[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 572754907 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002597] x2apic enabled [ 0.003008] Switched APIC routing to physical x2apic. [ 0.004017] kvm-guest: setup PV IPIs [ 0.007916] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008030] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009012] pid_max: default: 32768 minimum: 301 [ 0.010120] LSM: Security Framework initializing [ 0.011042] Yama: becoming mindful. [ 0.013028] SELinux: Initializing. [ 0.013994] *** VALIDATE selinux *** [ 0.021240] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.024000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.024140] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.025104] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.027054] *** VALIDATE tmpfs *** [ 0.028447] *** VALIDATE proc *** [ 0.029235] *** VALIDATE cgroup *** [ 0.030009] *** VALIDATE cgroup2 *** [ 0.031279] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.032136] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.033005] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.034022] Spectre V2 : User space: Vulnerable [ 0.035006] Speculative Store Bypass: Vulnerable [ 0.038550] debug: unmapping init [mem 0xffffffff8ee59000-0xffffffff8ee60fff] [ 0.040830] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.041886] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.042022] ... version: 2 [ 0.043012] ... bit width: 48 [ 0.044010] ... generic registers: 4 [ 0.045009] ... value mask: 0000ffffffffffff [ 0.046011] ... max period: 00007fffffffffff [ 0.047011] ... fixed-purpose events: 3 [ 0.048009] ... event mask: 000000070000000f [ 0.049252] rcu: Hierarchical SRCU implementation. [ 0.051543] smp: Bringing up secondary CPUs ... [ 0.052833] x86: Booting SMP configuration: [ 0.053054] .... node #0, CPUs: #1 #2 #3 [ 0.064516] smp: Brought up 1 node, 4 CPUs [ 0.066028] smpboot: Max logical packages: 1 [ 0.067014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.110302] node 0 deferred pages initialised in 41ms [ 0.117008] devtmpfs: initialized [ 0.118199] x86/mm: Memory block size: 128MB [ 0.120924] gcov: version magic: 0x41383552 [ 0.122055] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.123073] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.124216] pinctrl core: initialized pinctrl subsystem [ 0.125167] [ 0.125817] ************************************************************* [ 0.126048] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.127012] ** ** [ 0.128296] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.129010] ** ** [ 0.130011] ** This means that this kernel is built to expose internal ** [ 0.131014] ** IOMMU data structures, which may compromise security on ** [ 0.132010] ** your system. ** [ 0.133014] ** ** [ 0.134011] ** If you see this message and you are not debugging the ** [ 0.135011] ** kernel, report this immediately to your vendor! ** [ 0.136010] ** ** [ 0.137012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.138010] ************************************************************* [ 0.139808] NET: Registered protocol family 16 [ 0.140491] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.141051] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.142061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.143701] cpuidle: using governor menu [ 0.146534] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.150686] PCI: Using configuration type 1 for base access [ 0.153121] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.164335] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.166016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.169109] cryptd: max_cpu_qlen set to 1000 [ 0.170000] ACPI: Added _OSI(Module Device) [ 0.170000] ACPI: Added _OSI(Processor Device) [ 0.172013] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.175016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.181531] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.189881] ACPI: Interpreter enabled [ 0.190088] ACPI: PM: (supports S0 S3 S4 S5) [ 0.191028] ACPI: Using IOAPIC for interrupt routing [ 0.192083] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.193406] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.204783] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.205035] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.206015] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.207129] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.209481] acpiphp: Slot [2] registered [ 0.210212] acpiphp: Slot [5] registered [ 0.211182] acpiphp: Slot [6] registered [ 0.212126] acpiphp: Slot [7] registered [ 0.213102] acpiphp: Slot [8] registered [ 0.214135] acpiphp: Slot [9] registered [ 0.215212] acpiphp: Slot [10] registered [ 0.216098] acpiphp: Slot [3] registered [ 0.217121] acpiphp: Slot [4] registered [ 0.218090] acpiphp: Slot [11] registered [ 0.219099] acpiphp: Slot [12] registered [ 0.220248] acpiphp: Slot [13] registered [ 0.221221] acpiphp: Slot [14] registered [ 0.222082] acpiphp: Slot [15] registered [ 0.223151] acpiphp: Slot [16] registered [ 0.224077] acpiphp: Slot [17] registered [ 0.225157] acpiphp: Slot [18] registered [ 0.226375] acpiphp: Slot [19] registered [ 0.227225] acpiphp: Slot [20] registered [ 0.228072] acpiphp: Slot [21] registered [ 0.229104] acpiphp: Slot [22] registered [ 0.230103] acpiphp: Slot [23] registered [ 0.231109] acpiphp: Slot [24] registered [ 0.232085] acpiphp: Slot [25] registered [ 0.233083] acpiphp: Slot [26] registered [ 0.234080] acpiphp: Slot [27] registered [ 0.235203] acpiphp: Slot [28] registered [ 0.236092] acpiphp: Slot [29] registered [ 0.237153] acpiphp: Slot [30] registered [ 0.238142] acpiphp: Slot [31] registered [ 0.239049] PCI host bridge to bus 0000:00 [ 0.240019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.241018] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.242020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.243021] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.244021] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.245023] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.246223] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.248197] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.251012] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.264012] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.270060] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.273017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.276045] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.278016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.282692] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.286061] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.289045] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.293774] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.299012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.313059] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.319014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.326776] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.338015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.349016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.381000] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.397149] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.409014] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.419000] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.449016] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.467000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.477016] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.486014] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.505013] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.519039] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.532034] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.543018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.572000] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.591711] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.605016] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.618014] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.652021] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.668038] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.683014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.695020] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.724014] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.742425] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.747954] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.753129] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.758662] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.762388] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.771094] iommu: Default domain type: Passthrough [ 0.774383] SCSI subsystem initialized [ 0.776192] ACPI: bus type USB registered [ 0.777084] usbcore: registered new interface driver usbfs [ 0.778068] usbcore: registered new interface driver hub [ 0.782096] usbcore: registered new device driver usb [ 0.785431] pps_core: LinuxPPS API ver. 1 registered [ 0.787008] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.791068] PTP clock support registered [ 0.794035] EDAC MC: Ver: 3.0.0 [ 0.796210] PCI: Using ACPI for IRQ routing [ 0.798193] NetLabel: Initializing [ 0.800011] NetLabel: domain hash size = 128 [ 0.801009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.803140] NetLabel: unlabeled traffic allowed by default [ 0.805238] vgaarb: loaded [ 0.807544] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.810017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.817320] clocksource: Switched to clocksource kvm-clock [ 0.980668] VFS: Disk quotas dquot_6.6.0 [ 0.983772] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.989182] *** VALIDATE ramfs *** [ 0.992203] *** VALIDATE hugetlbfs *** [ 0.995678] pnp: PnP ACPI init [ 1.001289] pnp: PnP ACPI: found 6 devices [ 1.024710] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.029837] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.032137] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.035463] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.039418] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.042367] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.046076] NET: Registered protocol family 2 [ 1.049354] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.055383] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.061887] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.068866] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.073747] TCP: Hash tables configured (established 65536 bind 65536) [ 1.078643] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.084278] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.089731] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.095278] NET: Registered protocol family 1 [ 1.099403] RPC: Registered named UNIX socket transport module. [ 1.102839] RPC: Registered udp transport module. [ 1.105843] RPC: Registered tcp transport module. [ 1.110695] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.112928] NET: Registered protocol family 44 [ 1.114310] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.116993] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.121642] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.123794] PCI: CLS 0 bytes, default 64 [ 1.126767] Unpacking initramfs... [ 2.752803] debug: unmapping init [mem 0xffff8ea87cc54000-0xffff8ea87ffbffff] [ 2.758057] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.760637] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.763848] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.301710] Initialise system trusted keyrings [ 3.302860] Key type blacklist registered [ 3.304588] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.323789] zbud: loaded [ 3.331866] *** VALIDATE nfs *** [ 3.333219] *** VALIDATE nfs4 *** [ 3.334967] pstore: using deflate compression [ 3.339725] Platform Keyring initialized [ 3.491047] NET: Registered protocol family 38 [ 3.493817] Key type asymmetric registered [ 3.495455] Asymmetric key parser 'x509' registered [ 3.499452] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.504291] io scheduler mq-deadline registered [ 3.506533] io scheduler kyber registered [ 3.507912] io scheduler bfq registered [ 3.512648] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.516761] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.520885] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.524630] ACPI: Power Button [PWRF] [ 3.533288] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.540806] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.563140] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.583383] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.626701] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.657540] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.686600] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.691641] Non-volatile memory driver v1.3 [ 3.693454] Linux agpgart interface v0.103 [ 3.723196] virtio_blk virtio1: [vda] 149944 512-byte logical blocks (76.8 MB/73.2 MiB) [ 3.726062] vda: detected capacity change from 0 to 76771328 [ 3.745188] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.747763] vdb: detected capacity change from 0 to 1073741824 [ 3.769152] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.772311] vdc: detected capacity change from 0 to 2621440000 [ 3.805561] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.811756] vdd: detected capacity change from 0 to 2621440000 [ 3.837978] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.843289] vde: detected capacity change from 0 to 4294967296 [ 3.857839] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.860276] vdf: detected capacity change from 0 to 4294967296 [ 3.866331] libphy: Fixed MDIO Bus: probed [ 3.873681] usbcore: registered new interface driver usbserial_generic [ 3.879187] usbserial: USB Serial support registered for generic [ 3.882942] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.888651] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.891445] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.895056] mousedev: PS/2 mouse device common for all mice [ 3.899559] rtc_cmos 00:05: RTC can wake from S4 [ 3.912382] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.916180] rtc_cmos 00:05: registered as rtc0 [ 3.920439] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.920604] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.923744] intel_pstate: CPU model not supported [ 3.930636] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.939789] hid: raw HID events driver (C) Jiri Kosina [ 3.942720] usbcore: registered new interface driver usbhid [ 3.946708] usbhid: USB HID core driver [ 3.949072] drop_monitor: Initializing network drop monitor service [ 3.952491] Initializing XFRM netlink socket [ 3.955312] NET: Registered protocol family 10 [ 3.959240] Segment Routing with IPv6 [ 3.961565] NET: Registered protocol family 17 [ 3.964224] mpls_gso: MPLS GSO support [ 3.970558] RAS: Correctable Errors collector initialized. [ 3.973912] AVX version of gcm_enc/dec engaged. [ 3.976678] AES CTR mode by8 optimization enabled [ 4.062443] sched_clock: Marking stable (4062428568, 0)->(5305734976, -1243306408) [ 4.066208] registered taskstats version 1 [ 4.069222] Loading compiled-in X.509 certificates [ 4.071572] zswap: loaded using pool lzo/zbud [ 4.101940] Key type big_key registered [ 4.113961] Key type encrypted registered [ 4.122503] ima: No TPM chip found, activating TPM-bypass! [ 4.125544] ima: Allocated hash algorithm: sha1 [ 4.127151] ima: No architecture policies found [ 4.129202] evm: Initialising EVM extended attributes: [ 4.131182] evm: security.selinux [ 4.132540] evm: security.ima [ 4.133565] evm: security.capability [ 4.134776] evm: HMAC attrs: 0x1 [ 4.137370] rtc_cmos 00:05: setting system clock to 2026-09-16 22:01:02 UTC (1789596062) [ 4.146091] debug: unmapping init [mem 0xffffffff8fe03000-0xffffffff8fffffff] [ 4.149136] debug: unmapping init [mem 0xffffffff8eb82000-0xffffffff8ee58fff] [ 4.158398] Write protecting the kernel read-only data: 28672k [ 4.161659] debug: unmapping init [mem 0xffffffff8d203000-0xffffffff8d3fffff] [ 4.164545] debug: unmapping init [mem 0xffffffff8db14000-0xffffffff8dbfffff] [ 4.199153] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 4.208435] systemd[1]: Detected virtualization kvm. [ 4.210688] systemd[1]: Detected architecture x86-64. [ 4.212771] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.241127] systemd[1]: No hostname configured. [ 4.243687] systemd[1]: Set hostname to . [ 4.245659] random: systemd: uninitialized urandom read (16 bytes read) [ 4.248120] systemd[1]: Initializing machine ID from random generator. [ 4.336387] random: ln: uninitialized urandom read (6 bytes read) [ 4.476695] random: systemd: uninitialized urandom read (16 bytes read) [ 4.479754] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 4.483505] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.487945] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Reached target Local Encrypted Volumes. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 5.339882] device-mapper: uevent: version 1.0.3 [ 5.342194] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 6.376069] virtio_net virtio0 ens2: renamed from eth0 [ 6.416417] random: fast init done [ 6.582819] scsi host0: ata_piix [ 6.629223] scsi host1: ata_piix [ 6.630285] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 6.637895] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 11.640710] random: crng init done [ 11.641993] random: 7 urandom warning(s) missed due to ratelimiting [ 12.405238] dracut-initqueue[599]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 13.913206] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.711478] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.124917] SELinux: Disabled at runtime. [ 16.185833] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.196771] systemd[1]: Detected virtualization kvm. [ 16.198780] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.028155] systemd[1]: initrd-switch-root.service: Succeeded. [ 17.031710] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 17.039512] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 17.046223] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 17.050978] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 17.062049] systemd[1]: Starting Journal Service... Starting Journal Service... [ 17.068532] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. Mounting POSIX Message Queue File System... [ 17.124696] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on udev Control Socket. [ OK ] Created slice system-getty.slice. Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Huge Pages File System... [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Huge Pages File System. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 17.913471] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 18.647769] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 18.700034] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 18.999491] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 19.128558] EDAC sbridge: Ver: 1.1.2 [ 22.269326] Key type dns_resolver registered [ 22.688510] NFS: Registering the id_resolver key type [ 22.691316] Key type id_resolver registered [ 22.693551] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. Starting Network Manager... Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg652-server login: [ 42.434123] hrtimer: interrupt took 5110119 ns [ 85.406188] libcfs: loading out-of-tree module taints kernel. [ 85.463707] Key type ._llcrypt registered [ 85.465793] Key type .llcrypt registered [ 85.605728] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 107.457822] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 108.821353] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 108.849117] alg: No test for adler32 (adler32-zlib) [ 110.235924] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 111.542918] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 113.351214] Key type lgssc registered [ 114.536884] Lustre: Echo OBD driver; http://www.lustre.org/ [ 130.913962] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 169.324368] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 182.053293] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 182.118859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 183.362432] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 183.387799] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 183.481254] Lustre: lustre-MDT0000: new disk, initializing [ 183.571742] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 183.593902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 187.984740] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 200.115597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 200.239546] Lustre: 6530:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 200.284958] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 200.291562] Lustre: Skipped 1 previous similar message [ 200.400514] Lustre: lustre-MDT0001: new disk, initializing [ 200.476232] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 200.537614] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 200.544485] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 204.718507] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 209.581768] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 218.739336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 219.048357] Lustre: lustre-OST0000: new disk, initializing [ 219.052691] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 219.058877] Lustre: 8467:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 219.109543] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 224.797624] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 224.821361] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 224.827596] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 224.907348] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 238.114878] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 238.196765] Lustre: lustre-OST0001: new disk, initializing [ 238.199773] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 238.204589] Lustre: 9538:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 238.252046] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 244.290661] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 246.279412] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 246.296903] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 246.334315] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 257.140926] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 266.554797] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 273.447339] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing check_logdir /tmp/testlogs/ [ 278.445055] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing yml_node [ 283.456241] Lustre: DEBUG MARKER: Client: 2.17.58.39 [ 285.982345] Lustre: DEBUG MARKER: MDS: 2.17.58.39 [ 288.737251] Lustre: DEBUG MARKER: OSS: 2.17.58.39 [ 290.524196] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Wed Sep 16 18:05:48 EDT 2026 [ 306.955143] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 308.497213] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 328.161472] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 328.162940] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 328.172931] Lustre: Skipped 1 previous similar message [ 328.183043] Lustre: Skipped 3 previous similar messages [ 333.281973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 333.288076] Lustre: Skipped 3 previous similar messages [ 337.887456] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 338.073989] Lustre: server umount lustre-MDT0000 complete [ 341.638952] LustreError: 7487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789596400 with bad export cookie 10142272587126909871 [ 341.642306] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 341.652987] LustreError: 7487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 343.521287] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.524408] LustreError: 10114:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 343.525202] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 343.552538] Lustre: Skipped 3 previous similar messages [ 343.588121] LustreError: 10114:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 346.529587] LustreError: 10114:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 346.529681] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 346.562215] LustreError: 10114:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 346.588936] Lustre: Skipped 2 previous similar messages [ 348.176219] Lustre: server umount lustre-MDT0001 complete [ 351.708541] Lustre: server umount lustre-OST0000 complete [ 355.413314] Lustre: server umount lustre-OST0001 complete [ 363.209238] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 373.688597] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 409.708514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 409.964514] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 409.992438] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 410.070686] Lustre: lustre-MDT0000: new disk, initializing [ 410.157311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 410.176132] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 415.430940] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 426.377653] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 426.557658] Lustre: 15096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 426.664966] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 426.668859] Lustre: Skipped 1 previous similar message [ 426.771950] Lustre: lustre-MDT0001: new disk, initializing [ 426.895675] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 426.950406] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 426.967572] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 432.373821] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 441.834481] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 448.332531] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 456.004124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 456.482964] Lustre: lustre-OST0000: new disk, initializing [ 456.488693] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 456.497679] Lustre: 17064:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 456.608367] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 457.779901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 457.794204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 457.925475] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 463.162669] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 473.283633] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 478.858462] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 479.095931] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 484.000089] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 484.237266] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 488.932683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 488.934957] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 488.957259] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 488.982865] Lustre: Skipped 1 previous similar message [ 492.444506] Lustre: server umount lustre-OST0000 complete [ 498.657606] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 498.670982] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 498.680951] Lustre: Skipped 1 previous similar message [ 498.684456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 502.228976] Lustre: server umount lustre-MDT0000 complete [ 504.294959] LustreError: 15103:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 504.313589] LustreError: 15103:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 505.667416] LustreError: 15088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789596564 with bad export cookie 10142272587126911404 [ 505.667919] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 505.677373] LustreError: 15088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 506.061490] Lustre: server umount lustre-MDT0001 complete [ 511.952831] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:09:29 (1789596569) [ 519.139410] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 519.845324] LustreError: 19330:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.943587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 524.830801] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 525.290689] LustreError: 19331:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 534.433841] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 540.248926] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 548.513398] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 554.398477] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 561.111829] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 568.100825] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 576.161590] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 583.129714] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 590.306701] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 590.314663] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 590.324113] Lustre: Skipped 3 previous similar messages [ 590.332335] Lustre: Skipped 4 previous similar messages [ 596.189965] Lustre: server umount lustre-MDT0000 complete [ 600.233786] LustreError: 19310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789596658 with bad export cookie 10142272587126912286 [ 600.257370] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 600.262177] LustreError: 19310:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 600.544381] LustreError: 20837:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 600.545291] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 600.556599] LustreError: 20837:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 600.676942] Lustre: server umount lustre-MDT0001 complete [ 606.820888] Lustre: server umount lustre-OST0000 complete [ 628.160579] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 639.104899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 639.487257] LustreError: 23711:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 639.588188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 639.605340] Lustre: Skipped 2 previous similar messages [ 644.307280] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 653.905503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 659.582495] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 662.704594] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 669.388612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 676.543906] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 684.299423] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 684.444241] Lustre: lustre-OST0001: new disk, initializing [ 684.447509] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 684.453486] Lustre: 26026:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 685.884857] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 685.912188] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 685.962437] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 690.977577] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 700.672600] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 703.764622] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 715.747508] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 715.750839] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 715.787288] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 715.796074] Lustre: Skipped 3 previous similar messages [ 721.368355] Lustre: server umount lustre-MDT0000 complete [ 721.893587] LustreError: 23712:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 721.932849] LustreError: 23712:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 724.889023] LustreError: 23692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789596783 with bad export cookie 10142272587126913133 [ 724.893351] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 724.906494] LustreError: 23692:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 725.363979] Lustre: server umount lustre-MDT0001 complete [ 738.826988] Lustre: server umount lustre-OST0000 complete [ 751.837314] Lustre: server umount lustre-OST0001 complete [ 768.486531] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 778.435123] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 778.896445] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 778.903265] Lustre: Skipped 3 previous similar messages [ 783.019841] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 789.474957] LustreError: 29047:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 789.492260] LustreError: 29047:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 791.679983] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 796.385749] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 798.897073] Lustre: 30196:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 804.703376] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 811.688202] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 821.390449] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 828.709467] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 836.859131] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 840.525300] Lustre: 32073:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 848.352640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 848.364551] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 848.386517] Lustre: Skipped 3 previous similar messages [ 848.403514] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 848.417673] Lustre: Skipped 4 previous similar messages [ 852.879056] Lustre: server umount lustre-MDT0000 complete [ 856.624405] LustreError: 29029:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789596914 with bad export cookie 10142272587126914428 [ 856.648808] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 857.016492] Lustre: server umount lustre-MDT0001 complete [ 871.530719] Lustre: server umount lustre-OST0000 complete [ 874.719977] Lustre: 3666:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789596917/real 1789596917] req@ffff8ea8f9db5180 x1876527592792704/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789596933 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 875.126797] Lustre: server umount lustre-OST0001 complete [ 883.346678] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:15:41 (1789596941) [ 909.281113] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 919.303268] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 919.648374] LustreError: 34876:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 919.669490] LustreError: 34876:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 924.037862] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 933.239368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 938.631648] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 941.923576] Lustre: 36024:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 948.648413] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 955.395340] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 964.631373] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 972.003664] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 979.956182] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 983.415498] Lustre: 37905:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 990.176818] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 990.190292] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 990.204445] Lustre: Skipped 4 previous similar messages [ 990.216174] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 990.224333] Lustre: Skipped 4 previous similar messages [ 994.892573] Lustre: server umount lustre-MDT0000 complete [ 998.693729] LustreError: 37907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789597057 with bad export cookie 10142272587126915527 [ 998.704703] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 998.709479] LustreError: 37907:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 999.158457] Lustre: server umount lustre-MDT0001 complete [ 1012.859786] Lustre: server umount lustre-OST0000 complete [ 1027.299083] Lustre: server umount lustre-OST0001 complete [ 1035.505066] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:18:13 (1789597093) [ 1053.438489] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1064.367801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1064.898711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1064.906325] Lustre: Skipped 7 previous similar messages [ 1069.306569] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1077.542959] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1082.708083] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1085.722934] Lustre: 41200:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1091.544868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1098.147962] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1107.582054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1113.491628] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1119.687877] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1123.207520] Lustre: 43078:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1124.203776] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1124.208291] Lustre: Skipped 1 previous similar message [ 1129.440248] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1129.456568] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1129.476929] Lustre: Skipped 3 previous similar messages [ 1133.396419] Lustre: server umount lustre-MDT0000 complete [ 1136.769122] LustreError: 40031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789597195 with bad export cookie 10142272587126916640 [ 1136.776211] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1136.787446] LustreError: 40031:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1143.189628] Lustre: server umount lustre-MDT0001 complete [ 1152.325196] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1162.208676] LustreError: 3662:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8ea7c36ca680 x1876527592893952/t0(0) o250->MGC192.168.206.152@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1166.834970] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1171.883892] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1174.405897] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1179.136143] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1180.177717] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1180.191926] Lustre: Skipped 1 previous similar message [ 1186.450186] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1187.893576] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1193.454578] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1195.043036] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1200.624751] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1205.742614] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1205.745341] LustreError: 45219:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1205.792066] LustreError: 45219:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 32 previous similar messages [ 1206.058898] Lustre: server umount lustre-MDT0000 complete [ 1209.726985] Lustre: server umount lustre-MDT0001 complete [ 1223.639997] Lustre: server umount lustre-OST0000 complete [ 1237.678210] Lustre: server umount lustre-OST0001 complete [ 1245.458380] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:21:43 (1789597303) [ 1261.030874] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1269.409993] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1274.025184] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1281.774584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1285.793223] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1288.071789] Lustre: 48833:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1293.376172] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1298.975343] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1306.599065] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1312.956890] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1319.963342] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1324.699903] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1336.806575] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1336.818748] Lustre: Skipped 19 previous similar messages [ 1338.091994] Lustre: server umount lustre-MDT0000 complete [ 1341.610717] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789597399 with bad export cookie 10142272587126918565 [ 1341.627452] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1341.631358] LustreError: 47662:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1341.906589] Lustre: server umount lustre-MDT0001 complete [ 1356.125528] Lustre: server umount lustre-OST0000 complete [ 1358.053185] Lustre: 3663:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789597400/real 1789597400] req@ffff8ea7c36c8380 x1876527592981760/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789597416 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1360.108242] Lustre: server umount lustre-OST0001 complete [ 1368.798727] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:23:46 (1789597426) [ 1371.355222] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1373.180903] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:23:51 (1789597431) [ 1383.385432] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 1391.833278] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1412.672483] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1412.888246] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1413.014887] Lustre: lustre-MDT0000: new disk, initializing [ 1413.151615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1417.067128] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1425.054724] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1430.933438] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1431.164818] Lustre: lustre-OST0000: new disk, initializing [ 1431.168593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1431.172451] Lustre: Skipped 1 previous similar message [ 1431.176720] Lustre: 54908:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1432.381233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1432.401118] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1432.436290] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1437.209779] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1446.737705] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1448.889702] Lustre: DEBUG MARKER: create test files [ 1478.624621] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1478.642530] Lustre: Skipped 14 previous similar messages [ 1489.887120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1490.217113] Lustre: server umount lustre-OST0000 complete [ 1493.892032] Lustre: server umount lustre-MDT0000 complete [ 1504.401193] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 1507.033563] Key type lgssc unregistered [ 1507.313404] LNet: 56535:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1507.322187] LNetError: 56535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1507.349752] LNet: Removed LNI 192.168.206.152@tcp [ 1508.177142] Key type .llcrypt unregistered [ 1508.180628] Key type ._llcrypt unregistered [ 1532.961162] Key type ._llcrypt registered [ 1532.963278] Key type .llcrypt registered [ 1533.065460] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1534.009676] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1534.028860] alg: No test for adler32 (adler32-zlib) [ 1535.058283] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 1535.232549] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 1536.903250] Key type lgssc registered [ 1538.022920] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1545.132820] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1545.163113] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1546.612418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1550.598581] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1556.846781] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1562.903268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1563.282830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1568.938516] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1571.518963] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1571.599705] Lustre: 59240:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1571.605270] Lustre: 59240:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1571.611043] Lustre: 59240:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1571.630150] Lustre: 59240:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1571.639747] Lustre: 59240:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1571.645666] Lustre: 59240:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1575.539869] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1579.987038] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1584.632836] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1591.958986] Lustre: server umount lustre-OST0000 complete [ 1605.767677] Lustre: server umount lustre-MDT0000 complete [ 1615.531682] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 1617.700712] Key type lgssc unregistered [ 1617.945482] LNet: 60850:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1617.955318] LNetError: 60850:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1617.974700] LNet: Removed LNI 192.168.206.152@tcp [ 1618.734808] Key type .llcrypt unregistered [ 1618.738664] Key type ._llcrypt unregistered [ 1644.647417] Key type ._llcrypt registered [ 1644.651587] Key type .llcrypt registered [ 1644.768564] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1645.753599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1645.769141] alg: No test for adler32 (adler32-zlib) [ 1646.840652] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 1647.033669] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 1648.695227] Key type lgssc registered [ 1649.719692] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1656.724319] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1656.737790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1658.192899] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1662.494868] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1668.591604] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1673.653613] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1673.874871] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1679.328156] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1683.128225] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1683.219545] Lustre: 63660:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1683.231867] Lustre: 63660:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1683.243425] Lustre: 63660:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1683.255495] Lustre: 63660:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1683.268873] Lustre: 63660:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1683.275809] Lustre: 63660:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1687.551037] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1693.018827] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1703.904161] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1703.912068] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1703.931567] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1708.114566] Lustre: server umount lustre-OST0000 complete [ 1712.435653] Lustre: server umount lustre-MDT0000 complete [ 1723.602775] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 1726.455245] Key type lgssc unregistered [ 1726.678747] LNet: 65215:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1726.702122] LNetError: 65215:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1726.730479] LNet: Removed LNI 192.168.206.152@tcp [ 1727.579543] Key type .llcrypt unregistered [ 1727.581184] Key type ._llcrypt unregistered [ 1743.798595] Key type ._llcrypt registered [ 1743.803768] Key type .llcrypt registered [ 1743.949023] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 1757.102473] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1758.243914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1758.279210] alg: No test for adler32 (adler32-zlib) [ 1759.354370] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 1759.590192] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 1761.255224] Key type lgssc registered [ 1762.196678] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1793.626645] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1793.655676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1794.827158] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1794.875681] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1794.947242] Lustre: lustre-MDT0000: new disk, initializing [ 1795.054960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1795.080685] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1799.565448] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1809.511079] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1809.640890] Lustre: 68361:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1809.696258] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1809.700183] Lustre: Skipped 1 previous similar message [ 1809.771678] Lustre: lustre-MDT0001: new disk, initializing [ 1809.881587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1809.939719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1809.949829] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1814.280889] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1823.611436] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1828.816186] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1835.904637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1836.309346] Lustre: lustre-OST0000: new disk, initializing [ 1836.318408] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1836.325783] Lustre: 70332:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1836.495481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1842.008163] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 1843.776315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1843.799439] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1843.948772] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1851.723088] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1856.615085] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1856.836632] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1861.895629] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1862.126168] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1864.674995] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1864.676222] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1864.703693] Lustre: Skipped 1 previous similar message [ 1864.704633] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1864.745175] Lustre: Skipped 1 previous similar message [ 1869.797134] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1869.805182] Lustre: Skipped 1 previous similar message [ 1870.544293] Lustre: server umount lustre-OST0000 complete [ 1874.916122] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1874.919574] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1874.928973] Lustre: Skipped 1 previous similar message [ 1880.036706] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1880.051498] Lustre: Skipped 2 previous similar messages [ 1885.152367] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1885.160605] Lustre: Skipped 1 previous similar message [ 1888.223432] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1888.442147] Lustre: server umount lustre-MDT0000 complete [ 1891.889315] LustreError: 68355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789597950 with bad export cookie 14931080257299044775 [ 1891.896183] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1891.902599] LustreError: 68355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1892.177770] Lustre: server umount lustre-MDT0001 complete [ 1899.863925] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:32:37 (1789597957) [ 1910.443966] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1947.265878] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 1957.338917] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 1991.645469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1991.895291] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1991.932923] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1992.047327] Lustre: lustre-MDT0000: new disk, initializing [ 1992.169883] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1992.185197] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1996.657858] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2006.707503] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2006.839538] Lustre: 76350:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2006.923527] LustreError: 76362:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2006.939957] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2006.945939] LustreError: 76362:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2006.959556] Lustre: Skipped 1 previous similar message [ 2007.057076] Lustre: lustre-MDT0001: new disk, initializing [ 2007.194761] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2007.284380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2007.295852] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2012.153496] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2022.232635] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2028.847175] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2036.208809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2036.602316] Lustre: lustre-OST0000: new disk, initializing [ 2036.605915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2036.612099] Lustre: 78323:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2036.717683] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2043.819373] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2045.481694] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2045.499448] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2045.613492] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2053.855376] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2058.596262] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2058.831756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2063.211676] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2063.410406] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2066.410357] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2066.414075] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2066.416153] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2066.421967] Lustre: Skipped 1 previous similar message [ 2066.457940] LustreError: Skipped 1 previous similar message [ 2071.521330] LustreError: 78771:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2071.554894] LustreError: 78771:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2071.657903] Lustre: server umount lustre-OST0000 complete [ 2076.675461] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2076.698433] Lustre: Skipped 1 previous similar message [ 2081.733216] Lustre: server umount lustre-MDT0000 complete [ 2081.760522] LustreError: 77158:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2081.777289] LustreError: 77158:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2085.773785] LustreError: 76343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789598144 with bad export cookie 14931080257299045657 [ 2085.778531] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2085.787435] LustreError: 76343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2086.134655] Lustre: server umount lustre-MDT0001 complete [ 2094.820363] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:35:52 (1789598152) [ 2101.499924] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2102.064487] LustreError: 80735:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2102.118589] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2106.849483] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2107.363960] LustreError: 80736:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2115.014594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2119.497667] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2126.671905] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2131.825744] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2138.222439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2138.554520] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2138.562577] Lustre: Skipped 1 previous similar message [ 2144.458892] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2151.974831] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2155.885775] Lustre: 83405:0:(genops.c:1773:obd_export_evict_by_uuid()) MGS: evicting 476e4774-c410-4041-9fc6-f71c728ccde2 at adminstrative request [ 2157.037260] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2157.049916] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2167.272159] Lustre: MGS: Client 476e4774-c410-4041-9fc6-f71c728ccde2 (at 0@lo) reconnecting [ 2167.286448] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xcf35da62c128608b to 0xcf35da62c128632b [ 2167.310770] Lustre: MGC192.168.206.152@tcp: Connection restored to 0@lo (at 0@lo) [ 2193.779734] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2193.985184] Lustre: lustre-OST0001: new disk, initializing [ 2193.988497] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2193.993187] Lustre: 83800:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2200.222071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2200.247246] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2200.364345] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2200.597900] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2208.851645] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2215.906049] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2215.928450] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2215.945367] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2215.956127] Lustre: Skipped 3 previous similar messages [ 2215.956539] Lustre: Skipped 1 previous similar message [ 2225.119369] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2225.243920] Lustre: server umount lustre-OST0001 complete [ 2226.148399] LustreError: 83031:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2226.169757] LustreError: 83031:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 2228.706689] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2228.718894] LustreError: Skipped 1 previous similar message [ 2228.725173] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2235.028831] Lustre: server umount lustre-OST0000 complete [ 2244.840956] Lustre: server umount lustre-MDT0000 complete [ 2246.625151] LustreError: 81784:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2246.656431] LustreError: 81784:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 2248.087335] LustreError: 80717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789598306 with bad export cookie 14931080257299047211 [ 2248.094363] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2248.098991] LustreError: 80717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2248.389525] Lustre: server umount lustre-MDT0001 complete [ 2257.081303] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:38:35 (1789598315) [ 2258.275591] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2260.091959] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:38:38 (1789598318) [ 2261.515347] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2263.447885] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:38:41 (1789598321) [ 2265.741729] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2267.889404] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:38:45 (1789598325) [ 2269.202467] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2271.044410] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:38:48 (1789598328) [ 2272.643933] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2278.659853] loop: module loaded [ 2278.677810] loop0: detected capacity change from 0 to 4294967296 [ 2278.720116] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2280.209305] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2283.662577] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2287.219874] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2309.914742] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2309.971764] LustreError: 87924:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2309.984344] LustreError: 87924:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2309.994333] LustreError: 87924:0:(tgt_mount.c:2581:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2310.004079] LustreError: 87924:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2326.522387] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 2335.807927] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 2369.168868] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2369.398941] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2369.439228] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2369.558550] Lustre: lustre-MDT0000: new disk, initializing [ 2369.645313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2369.652931] Lustre: Skipped 1 previous similar message [ 2369.672784] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2374.008453] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2383.641086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2383.790262] Lustre: 90788:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2384.109967] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2387.811772] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2395.366855] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2401.881573] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2408.409891] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2408.644300] Lustre: 92758:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2415.081891] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2417.214298] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2417.307289] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2423.631248] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2427.664164] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2427.872269] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2431.853359] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2432.085372] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2436.071295] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2436.076501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2436.092858] Lustre: Skipped 4 previous similar messages [ 2436.103664] Lustre: Skipped 9 previous similar messages [ 2448.351178] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2448.603371] Lustre: server umount lustre-OST0000 complete [ 2456.039996] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2458.871324] Lustre: server umount lustre-MDT0000 complete [ 2461.664543] LustreError: 90800:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2461.683271] LustreError: 90800:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 2462.911390] LustreError: 90782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789598521 with bad export cookie 14931080257299047827 [ 2462.916137] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2462.931347] LustreError: 90782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2463.337620] Lustre: server umount lustre-MDT0001 complete [ 2472.963336] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:42:10 (1789598530) [ 2480.169056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2485.972790] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2495.331209] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2501.059923] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2508.689488] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2514.683978] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2522.326404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2522.722350] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2522.733994] Lustre: Skipped 4 previous similar messages [ 2529.643385] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2537.249620] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2542.035409] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2542.295879] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2547.255410] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2547.530459] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2554.891103] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2555.097081] Lustre: lustre-OST0001: new disk, initializing [ 2555.102742] Lustre: Skipped 2 previous similar messages [ 2555.107591] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2555.112179] Lustre: Skipped 3 previous similar messages [ 2555.122636] Lustre: 98541:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2561.634945] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2564.204853] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2564.217226] Lustre: Skipped 2 previous similar messages [ 2564.229550] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2564.333160] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2571.500425] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2576.086342] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2576.280167] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2580.440708] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2580.665698] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2597.608823] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2618.951745] Lustre: Failing over lustre-MDT0000 [ 2619.448518] Lustre: server umount lustre-MDT0000 complete [ 2620.904036] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2620.912745] LustreError: 95164:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2620.916944] Lustre: Skipped 4 previous similar messages [ 2620.942025] LustreError: 95164:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 2636.751708] Lustre: 66579:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789598679/real 1789598679] req@00000000adee7e1b x1876529320828544/t0(0) o400->MGC192.168.206.152@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789598695 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2636.787743] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2642.773029] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2642.776333] LDISKFS-fs (dm-0): recovery complete [ 2642.785300] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2647.471551] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2649.166198] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2649.167393] Lustre: 95165:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2651.385786] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2652.644246] Lustre: 95458:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2652.662849] Lustre: 95458:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 1 previous similar message [ 2652.673730] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2652.758828] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2653.079207] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2653.405771] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2653.419353] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2653.759255] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2653.766345] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 1 previous similar message [ 2653.772259] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2653.778431] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 7 previous similar messages [ 2654.417055] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2654.422891] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 2 previous similar messages [ 2655.071104] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2655.076489] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 3 previous similar messages [ 2656.041577] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2656.051340] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 6 previous similar messages [ 2656.690730] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2656.706763] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2657.343110] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2657.350688] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 6 previous similar messages [ 2660.208343] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2660.234402] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 12 previous similar messages [ 2660.898230] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2660.922499] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2661.567796] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2661.576535] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 12 previous similar messages [ 2668.252896] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2668.260785] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 24 previous similar messages [ 2669.208199] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2669.219983] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2669.887258] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2669.894383] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 25 previous similar messages [ 2684.436639] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2684.462201] Lustre: 101389:0:(ldlm_lib.c:2119:extend_recovery_timer()) Skipped 50 previous similar messages [ 2685.461385] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2685.475540] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2686.111198] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2686.116983] LustreError: 101389:0:(ldlm_lib.c:2732:replay_request_or_update()) Skipped 50 previous similar messages [ 2709.232155] Lustre: 101389:0:(ldlm_lib.c:2428:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2709.240786] Lustre: 101389:0:(ldlm_lib.c:2438:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2709.247818] Lustre: 101389:0:(genops.c:1600:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client b70f5d74-65b9-41e6-be50-27eecf1e4324@192.168.206.52@tcp [ 2709.257215] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2709.260344] Lustre: 101389:0:(ldlm_lib.c:1948:abort_req_replay_queue()) @@@ aborted: req@00000000a19fcb11 x1876529308044544/t0(8589934774) o35->b70f5d74-65b9-41e6-be50-27eecf1e4324@192.168.206.52@tcp:383/0 lens 392/0 e 0 to 0 dl 1789598778 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2709.280535] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2709.297592] Lustre: lustre-MDT0000: Denying connection for new client b70f5d74-65b9-41e6-be50-27eecf1e4324 (at 192.168.206.52@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2709.310454] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2709.373924] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2709.388541] Lustre: Skipped 2 previous similar messages [ 2709.392367] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2709.443215] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 2709.446406] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 2722.288453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2722.292057] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2722.297602] Lustre: Skipped 2 previous similar messages [ 2722.304925] Lustre: Skipped 7 previous similar messages [ 2727.450160] Lustre: server umount lustre-OST0000 complete [ 2745.311713] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2745.607107] Lustre: server umount lustre-OST0001 complete [ 2750.437412] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2755.565384] Lustre: server umount lustre-MDT0000 complete [ 2758.125305] LustreError: 95164:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2758.152344] LustreError: 95164:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 39 previous similar messages [ 2759.460336] LustreError: 95149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789598817 with bad export cookie 14931080257299099039 [ 2759.470658] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2759.474317] LustreError: 95149:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2759.811471] Lustre: server umount lustre-MDT0001 complete [ 2768.777940] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:47:06 (1789598826) [ 2777.278612] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2777.282651] Lustre: *** cfs_fail_loc=197, val=0*** [ 2792.799745] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2797.919710] LustreError: 103365:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.152@tcp: failed processing log, type 4: rc = -110 [ 2803.074114] Lustre: *** cfs_fail_loc=197, val=0*** [ 2803.077416] Lustre: Skipped 65 previous similar messages [ 2823.647530] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2823.658323] Lustre: Skipped 2 previous similar messages [ 2829.720666] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2837.410177] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2839.461803] Lustre: server umount lustre-OST0000 complete [ 2848.325144] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:48:26 (1789598906) [ 2870.610188] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2875.659400] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2884.333253] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2887.587360] LustreError: 105017:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 2887.602297] LustreError: 105017:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 2887.608520] LustreError: 105017:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 16, retries 0, failed: rc = -5 [ 2887.830178] Lustre: server umount lustre-MDT0000 complete [ 2920.198385] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 2930.513665] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 2961.801859] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2962.038577] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2962.072411] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2962.166337] Lustre: lustre-MDT0000: new disk, initializing [ 2962.250850] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2966.352607] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2975.007771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2975.140039] Lustre: 109193:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2975.369375] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2979.653423] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 2989.051947] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2995.313940] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3002.358991] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3002.677788] Lustre: 111161:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3009.313278] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3011.718876] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3019.426462] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3024.357821] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3024.518373] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3029.341799] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3029.551935] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3032.032221] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3032.045810] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3032.066981] Lustre: Skipped 4 previous similar messages [ 3032.082727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3032.100239] Lustre: Skipped 9 previous similar messages [ 3037.674144] LustreError: 111144:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3037.692197] LustreError: 111144:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 3037.822424] Lustre: server umount lustre-OST0000 complete [ 3047.778613] Lustre: server umount lustre-MDT0000 complete [ 3051.053332] LustreError: 109184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789599109 with bad export cookie 14931080257299109679 [ 3051.065837] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3051.081774] LustreError: 109184:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3051.523695] Lustre: server umount lustre-MDT0001 complete [ 3060.305699] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:51:58 (1789599118) [ 3069.777339] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 3072.618545] Key type lgssc unregistered [ 3072.859443] LNet: 113648:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3072.863530] LNetError: 113648:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3072.878297] LNet: Removed LNI 192.168.206.152@tcp [ 3073.487896] Key type .llcrypt unregistered [ 3073.490936] Key type ._llcrypt unregistered [ 3090.789644] Key type ._llcrypt registered [ 3090.796585] Key type .llcrypt registered [ 3090.931602] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 3106.336835] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 3107.668743] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3107.726071] alg: No test for adler32 (adler32-zlib) [ 3108.768178] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 3109.020054] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 3110.796533] Key type lgssc registered [ 3112.044502] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3154.954535] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3154.983469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3156.243943] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3156.287207] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3156.398460] Lustre: lustre-MDT0000: new disk, initializing [ 3156.568290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3156.605102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3161.724197] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3173.425048] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3173.741241] Lustre: lustre-OST0000: new disk, initializing [ 3173.744789] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3173.749035] Lustre: Skipped 1 previous similar message [ 3173.755572] Lustre: 117816:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3173.836729] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3179.136593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3179.141660] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3179.190230] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3179.478381] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3188.721166] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3188.873411] Lustre: lustre-OST0001: new disk, initializing [ 3188.878897] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3188.887532] Lustre: 118679:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3188.979098] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3194.852630] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3197.055077] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3197.064660] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3197.101711] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3207.137281] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3207.143200] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3207.155986] Lustre: Skipped 1 previous similar message [ 3207.184595] Lustre: Skipped 1 previous similar message [ 3209.276546] Lustre: server umount lustre-MDT0000 complete [ 3214.759078] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3218.472447] LustreError: 119288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789599276 with bad export cookie 10059814896604490193 [ 3218.480054] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3218.485451] LustreError: 119288:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3218.563751] Lustre: server umount lustre-OST0000 complete [ 3221.481523] Lustre: server umount lustre-OST0001 complete [ 3228.752813] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 18:54:46 (1789599286) [ 3248.264769] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 18:55:06 (1789599306) [ 3258.675384] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 3281.802774] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:55:39 (1789599339) [ 3317.041317] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 3326.297195] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 3361.813883] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3362.234553] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3362.266947] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3362.401491] Lustre: lustre-MDT0000: new disk, initializing [ 3362.542817] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3362.589521] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3367.660333] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3378.485779] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3378.743792] Lustre: 126431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3378.845716] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3378.854544] Lustre: Skipped 1 previous similar message [ 3379.016780] Lustre: lustre-MDT0001: new disk, initializing [ 3379.150401] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3379.196307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3379.208176] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3384.293603] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3394.888472] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3400.657854] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3407.276215] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3407.615407] Lustre: lustre-OST0000: new disk, initializing [ 3407.618854] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3407.627485] Lustre: 128397:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3407.750309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3413.998622] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3416.632945] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3416.645080] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3416.784575] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3422.913932] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3427.290713] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3427.515466] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3431.729693] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3431.938712] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3437.537804] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3437.550598] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3437.555911] LustreError: Skipped 1 previous similar message [ 3437.564449] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3437.588387] Lustre: Skipped 1 previous similar message [ 3437.618522] Lustre: Skipped 1 previous similar message [ 3442.665351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3442.668787] Lustre: Skipped 1 previous similar message [ 3447.778360] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3447.789391] Lustre: Skipped 1 previous similar message [ 3448.287113] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3448.465414] Lustre: server umount lustre-OST0000 complete [ 3452.906989] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3452.911052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3452.934535] Lustre: Skipped 1 previous similar message [ 3452.954063] Lustre: Skipped 1 previous similar message [ 3463.136733] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3463.146913] Lustre: Skipped 3 previous similar messages [ 3466.209558] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3466.355520] Lustre: server umount lustre-MDT0000 complete [ 3469.989621] LustreError: 126422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789599528 with bad export cookie 10059814896604491418 [ 3469.992226] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3470.001915] LustreError: 126422:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3470.547754] Lustre: server umount lustre-MDT0001 complete [ 3477.976614] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:58:56 (1789599536) [ 3483.555565] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3483.988559] LustreError: 130812:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3484.010489] LustreError: 130812:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 3484.079515] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3488.785403] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3489.250875] LustreError: 130813:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3489.278541] LustreError: 130813:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 3494.373393] LustreError: 130812:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3497.966516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3502.326796] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3509.328478] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3514.918092] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3521.838631] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3522.208273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3522.213285] Lustre: Skipped 1 previous similar message [ 3529.740239] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3538.755817] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3542.763196] Lustre: *** cfs_fail_loc=159, val=0*** [ 3559.058320] Lustre: lustre-MDT0000: Client 5187772c-d28f-4279-ae8a-b43149377e54 (at 192.168.206.52@tcp) reconnecting [ 3561.319051] Lustre: *** cfs_fail_loc=159, val=0*** [ 3561.320761] Lustre: Skipped 5 previous similar messages [ 3576.993296] Lustre: lustre-MDT0000: Client 5187772c-d28f-4279-ae8a-b43149377e54 (at 192.168.206.52@tcp) reconnecting [ 3583.864730] Lustre: server umount lustre-OST0000 complete [ 3590.628700] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3590.636288] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3590.671680] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3601.375123] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3601.375413] Lustre: 115023:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789599643/real 1789599643] req@00000000105ea6ef x1876530736245632/t0(0) o13->lustre-OST0000-osc-MDT0001@0@lo:7/4 lens 224/368 e 0 to 1 dl 1789599659 ref 1 fl Rpc:XQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3601.397125] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3601.405197] Lustre: Skipped 1 previous similar message [ 3601.567524] Lustre: server umount lustre-MDT0000 complete [ 3602.272373] Lustre: 115022:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789599644/real 1789599644] req@00000000f3582e64 x1876530736246784/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1789599660 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3606.364530] LustreError: 130793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789599664 with bad export cookie 10059814896604492300 [ 3606.378928] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3606.382317] LustreError: 130793:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3606.848082] Lustre: server umount lustre-MDT0001 complete [ 3618.460409] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 3621.499291] Key type lgssc unregistered [ 3621.746264] LNet: 134602:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3621.758781] LNetError: 134602:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3621.788984] LNet: Removed LNI 192.168.206.152@tcp [ 3622.641420] Key type .llcrypt unregistered [ 3622.643481] Key type ._llcrypt unregistered [ 3648.925476] Key type ._llcrypt registered [ 3648.928958] Key type .llcrypt registered [ 3649.058802] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 3650.658254] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3650.734583] alg: No test for adler32 (adler32-zlib) [ 3651.850741] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 3652.155719] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 3653.809494] Key type lgssc registered [ 3655.262941] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3663.392275] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3663.431361] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3664.930055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3669.306510] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3675.110417] LustreError: 136016:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3675.131444] LustreError: 136016:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3677.870143] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3678.327492] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3682.866787] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3690.015573] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3695.367771] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3701.541375] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3701.863861] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3707.632931] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3710.060561] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3713.759383] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3723.652770] Lustre: *** cfs_fail_loc=159, val=0*** [ 3740.260252] Lustre: lustre-MDT0000: Client 279b0d42-c681-47b0-a708-410529879de4 (at 192.168.206.52@tcp) reconnecting [ 3742.327644] Lustre: *** cfs_fail_loc=159, val=0*** [ 3757.690133] Lustre: lustre-MDT0001: Client 279b0d42-c681-47b0-a708-410529879de4 (at 192.168.206.52@tcp) reconnecting [ 3759.251817] Lustre: *** cfs_fail_loc=159, val=0*** [ 3759.260675] Lustre: Skipped 4 previous similar messages [ 3775.629068] Lustre: lustre-MDT0001: Client 279b0d42-c681-47b0-a708-410529879de4 (at 192.168.206.52@tcp) reconnecting [ 3781.227933] Lustre: 139052:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3784.067389] Lustre: *** cfs_fail_loc=159, val=0*** [ 3784.068878] Lustre: Skipped 3 previous similar messages [ 3799.715493] Lustre: lustre-MDT0000: Client c1fa5ff9-3a0f-4cf3-801e-ebdf72382ce2 (at 192.168.206.52@tcp) reconnecting [ 3801.539331] Lustre: *** cfs_fail_loc=159, val=0*** [ 3801.547085] Lustre: Skipped 14 previous similar messages [ 3817.072891] Lustre: lustre-MDT0000: Client c1fa5ff9-3a0f-4cf3-801e-ebdf72382ce2 (at 192.168.206.52@tcp) reconnecting [ 3818.801744] Lustre: 139346:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3825.137261] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 3825.148068] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3825.149728] LustreError: Skipped 2 previous similar messages [ 3825.197448] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3828.195574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3828.204860] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3828.204860] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3828.216034] LustreError: Skipped 3 previous similar messages [ 3829.945335] Lustre: server umount lustre-OST0000 complete [ 3833.318869] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3833.320541] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3833.330560] Lustre: Skipped 1 previous similar message [ 3833.334280] Lustre: Skipped 1 previous similar message [ 3838.435662] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3838.443045] Lustre: Skipped 2 previous similar messages [ 3839.700689] Lustre: server umount lustre-MDT0000 complete [ 3843.310692] LustreError: 138828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789599901 with bad export cookie 11564587521855667607 [ 3843.312250] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3843.321577] LustreError: 138828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3843.760621] Lustre: server umount lustre-MDT0001 complete [ 3856.709475] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 3859.277684] Key type lgssc unregistered [ 3859.571698] LNet: 140399:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3859.579476] LNetError: 140399:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3859.608282] LNet: Removed LNI 192.168.206.152@tcp [ 3860.574335] Key type .llcrypt unregistered [ 3860.575983] Key type ._llcrypt unregistered [ 3885.165010] Key type ._llcrypt registered [ 3885.167850] Key type .llcrypt registered [ 3885.286918] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 3886.315712] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3886.573488] alg: No test for adler32 (adler32-zlib) [ 3887.746670] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 3888.068467] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 3889.775211] Key type lgssc registered [ 3891.075479] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3898.840783] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3898.868862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3900.269369] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3905.022227] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3910.626725] LustreError: 141812:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3910.645240] LustreError: 141812:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 3913.263969] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3913.656363] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3917.968317] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3924.851194] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3930.346365] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3937.253136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3937.587611] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3938.632916] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 3943.376731] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 3944.758654] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 3950.094426] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3956.341042] Lustre: 144394:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3958.990552] Lustre: 144442:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3960.289374] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 3961.053703] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 3965.102991] Lustre: 144588:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 3965.927827] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 3966.821357] Lustre: 144686:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3970.534345] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3970.536766] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3970.566665] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3970.582872] Lustre: Skipped 1 previous similar message [ 3975.655810] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3975.664087] Lustre: Skipped 1 previous similar message [ 3980.768658] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3980.777093] Lustre: Skipped 1 previous similar message [ 3984.351637] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3984.532751] Lustre: server umount lustre-OST0000 complete [ 3990.496478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3990.513941] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3990.535739] Lustre: Skipped 1 previous similar message [ 3990.544056] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3996.136338] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3996.152444] Lustre: Skipped 3 previous similar messages [ 4002.271260] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4002.431329] Lustre: server umount lustre-MDT0000 complete [ 4005.896078] LustreError: 141794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789600064 with bad export cookie 7043285798013151637 [ 4005.904312] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4005.910888] LustreError: 141794:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4006.235283] Lustre: server umount lustre-MDT0001 complete [ 4016.230670] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 4019.034641] Key type lgssc unregistered [ 4019.348675] LNet: 145748:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4019.361581] LNetError: 145748:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4019.383131] LNet: Removed LNI 192.168.206.152@tcp [ 4020.183074] Key type .llcrypt unregistered [ 4020.185142] Key type ._llcrypt unregistered [ 4044.112459] Key type ._llcrypt registered [ 4044.116177] Key type .llcrypt registered [ 4044.203987] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4045.170847] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4045.231991] alg: No test for adler32 (adler32-zlib) [ 4046.234886] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 4046.506164] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 4048.175196] Key type lgssc registered [ 4049.280866] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4056.202652] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4056.238652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4057.606729] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4062.042930] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4066.787081] LustreError: 147160:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4066.798627] LustreError: 147160:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4069.752116] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4070.136841] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4074.413222] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4081.070985] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4086.269386] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4091.964171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4092.302306] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4095.415415] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4098.087860] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4103.603555] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4104.038174] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4110.720633] Lustre: *** cfs_fail_loc=159, val=0*** [ 4127.353235] Lustre: lustre-MDT0000: Client 3e2b94ee-bcd8-4bd9-833d-21d4f173c5f3 (at 192.168.206.52@tcp) reconnecting [ 4131.297053] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4131.303304] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4131.319252] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4132.850018] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4132.850078] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4132.858296] LustreError: Skipped 3 previous similar messages [ 4132.880648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4133.402462] Lustre: server umount lustre-OST0000 complete [ 4136.931919] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4136.936787] LustreError: Skipped 4 previous similar messages [ 4136.939962] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4136.948125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4139.487958] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4139.496083] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4144.612209] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4144.619459] Lustre: Skipped 2 previous similar messages [ 4151.264416] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4151.467487] Lustre: server umount lustre-MDT0000 complete [ 4153.824656] LustreError: 147156:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4153.840927] LustreError: 147156:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4154.695064] LustreError: 147139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789600213 with bad export cookie 4178692539964408762 [ 4154.702181] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4154.705669] LustreError: 147139:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4155.020776] Lustre: server umount lustre-MDT0001 complete [ 4163.461428] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 4165.970946] Key type lgssc unregistered [ 4166.270888] LNet: 150821:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4166.281555] LNetError: 150821:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4166.298131] LNet: Removed LNI 192.168.206.152@tcp [ 4167.183164] Key type .llcrypt unregistered [ 4167.185257] Key type ._llcrypt unregistered [ 4187.037334] Key type ._llcrypt registered [ 4187.039824] Key type .llcrypt registered [ 4187.149530] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4187.877323] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4187.989870] alg: No test for adler32 (adler32-zlib) [ 4188.982851] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 4189.198152] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 4190.839240] Key type lgssc registered [ 4191.861261] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4198.142758] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4198.155362] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4199.490201] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4203.084466] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4209.644982] LustreError: 152212:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4209.677518] LustreError: 152212:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4210.981803] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4211.342984] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4215.275214] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4221.176353] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4225.560166] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4232.375425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4232.746746] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4235.841429] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4239.481595] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4244.112728] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4247.552284] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4266.022447] Lustre: lustre-OST0000: evicting 7f4e4605-8a06-4243-940f-d9a9ce506cb8 (at 192.168.206.52@tcp) by administrative request [ 4266.032821] Lustre: Skipped 1 previous similar message [ 4295.649986] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4295.652386] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4295.668098] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4295.679870] Lustre: Skipped 1 previous similar message [ 4300.143755] Lustre: server umount lustre-OST0000 complete [ 4303.329064] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4303.338403] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4303.357033] Lustre: Skipped 1 previous similar message [ 4303.366721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4305.891149] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4305.892057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4305.920111] Lustre: Skipped 1 previous similar message [ 4309.596961] Lustre: server umount lustre-MDT0000 complete [ 4311.011433] LustreError: 152213:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4311.029321] LustreError: 152213:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4313.077649] LustreError: 152193:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789600371 with bad export cookie 2128497204180496676 [ 4313.086125] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4313.427620] Lustre: server umount lustre-MDT0001 complete [ 4324.386762] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 4327.179253] Key type lgssc unregistered [ 4327.447817] LNet: 156278:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4327.452970] LNetError: 156278:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4327.475866] LNet: Removed LNI 192.168.206.152@tcp [ 4328.176479] Key type .llcrypt unregistered [ 4328.178800] Key type ._llcrypt unregistered [ 4391.976244] Key type ._llcrypt registered [ 4391.978215] Key type .llcrypt registered [ 4392.060728] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4392.671886] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4392.704337] alg: No test for adler32 (adler32-zlib) [ 4393.748091] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 4394.035798] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 4395.719970] Key type lgssc registered [ 4396.677506] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4403.321700] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4403.336043] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4404.698984] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4408.485170] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4414.952592] LustreError: 158743:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4414.992980] LustreError: 158743:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4416.312935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4416.658941] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4421.910921] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4429.407230] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4436.536562] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4441.735969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4442.123639] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4447.361242] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4450.367061] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4454.484074] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4461.611435] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4471.787701] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4471.794469] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4471.826067] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4471.834427] LustreError: Skipped 1 previous similar message [ 4471.840128] Lustre: Skipped 1 previous similar message [ 4476.896074] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4476.907123] Lustre: Skipped 1 previous similar message [ 4477.239848] Lustre: server umount lustre-OST0000 complete [ 4482.017882] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4482.022724] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4482.028702] Lustre: Skipped 2 previous similar messages [ 4482.037384] Lustre: Skipped 1 previous similar message [ 4487.137264] LustreError: 158737:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4487.158824] LustreError: 158737:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4489.039818] Lustre: server umount lustre-MDT0000 complete [ 4492.258207] LustreError: 159473:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4492.282141] LustreError: 159473:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 4492.569386] LustreError: 158724:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789600550 with bad export cookie 1261725276504599468 [ 4492.586858] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4492.953891] Lustre: server umount lustre-MDT0001 complete [ 4504.115588] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 4506.898598] Key type lgssc unregistered [ 4507.253178] LNet: 162303:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4507.264507] LNetError: 162303:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4507.290880] LNet: Removed LNI 192.168.206.152@tcp [ 4508.341440] Key type .llcrypt unregistered [ 4508.344529] Key type ._llcrypt unregistered [ 4529.066902] Key type ._llcrypt registered [ 4529.069168] Key type .llcrypt registered [ 4529.165109] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4529.957378] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4530.001549] alg: No test for adler32 (adler32-zlib) [ 4531.042750] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 4531.321224] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 4533.015375] Key type lgssc registered [ 4534.012715] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4545.518983] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:16:43 (1789600603) [ 4559.718991] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 4569.119518] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4603.317887] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4603.344187] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4604.749727] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4604.786398] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4604.876926] Lustre: lustre-MDT0000: new disk, initializing [ 4605.031929] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4605.088260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4609.355257] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4620.349802] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4620.430337] Lustre: 166544:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4620.468482] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4620.471132] Lustre: Skipped 1 previous similar message [ 4620.527942] Lustre: lustre-MDT0001: new disk, initializing [ 4620.611109] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4620.648520] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4620.664385] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4624.758934] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4632.030838] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4636.514045] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4642.748081] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4642.998691] Lustre: lustre-OST0000: new disk, initializing [ 4643.003161] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4643.010209] Lustre: 168510:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4643.123356] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4648.515042] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4648.528707] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4648.633721] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4649.113786] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4657.507271] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4669.409219] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4669.414619] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4669.426546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4669.439975] Lustre: Skipped 1 previous similar message [ 4672.177966] Lustre: server umount lustre-OST0000 complete [ 4677.097800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4677.114478] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4677.125031] Lustre: Skipped 1 previous similar message [ 4677.136621] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4679.649660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4679.654241] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4681.731958] Lustre: server umount lustre-MDT0000 complete [ 4684.760484] LustreError: 166537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789600743 with bad export cookie 3463497071042473633 [ 4684.761690] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4684.772110] LustreError: 166537:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4685.114651] Lustre: server umount lustre-MDT0001 complete [ 4695.021487] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 4697.906483] Key type lgssc unregistered [ 4698.317080] LNet: 170413:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4698.330450] LNetError: 170413:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4698.347509] LNet: Removed LNI 192.168.206.152@tcp [ 4699.241377] Key type .llcrypt unregistered [ 4699.244562] Key type ._llcrypt unregistered [ 4724.855454] Key type ._llcrypt registered [ 4724.857202] Key type .llcrypt registered [ 4725.088038] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 4726.238817] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4726.306113] alg: No test for adler32 (adler32-zlib) [ 4727.593403] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 4727.795313] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 4729.487291] Key type lgssc registered [ 4730.654574] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4737.732571] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4737.751612] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4739.182328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4743.431909] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4748.774538] LustreError: 171824:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4748.795342] LustreError: 171824:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 4751.112725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.593935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4755.390767] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4763.113313] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4768.984561] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4775.397049] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4775.816280] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4782.026973] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 4784.068784] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4789.254133] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4795.784626] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4795.800331] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4795.816422] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4795.821674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4795.833917] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4796.748899] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4796.765703] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4796.785173] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4796.791862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4796.803919] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4797.666850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.345687] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4798.352661] Lustre: Skipped 1 previous similar message [ 4798.369482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4798.389661] Lustre: Skipped 1 previous similar message [ 4798.398042] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4798.400538] Lustre: Skipped 1 previous similar message [ 4798.405907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4798.430730] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4798.448418] Lustre: Skipped 1 previous similar message [ 4799.436321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4800.302921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.257358] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4801.267778] Lustre: Skipped 2 previous similar messages [ 4801.286564] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4801.300885] Lustre: Skipped 2 previous similar messages [ 4801.304815] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4801.309608] Lustre: Skipped 2 previous similar messages [ 4801.313253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4801.323554] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4801.330403] Lustre: Skipped 2 previous similar messages [ 4802.028167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4802.938669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4803.875854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4804.975859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.031374] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4806.036930] Lustre: Skipped 4 previous similar messages [ 4806.052265] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4806.061937] Lustre: Skipped 4 previous similar messages [ 4806.066577] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4806.071833] Lustre: Skipped 4 previous similar messages [ 4806.075107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4806.083834] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4806.089895] Lustre: Skipped 4 previous similar messages [ 4807.109064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.100067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4808.987482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4809.948423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4810.928457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.015487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4812.826805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4813.790075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.540918] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4814.559857] Lustre: Skipped 8 previous similar messages [ 4814.584768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4814.590274] Lustre: Skipped 8 previous similar messages [ 4814.593464] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4814.599599] Lustre: Skipped 8 previous similar messages [ 4814.604516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4814.641210] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4814.655161] Lustre: Skipped 8 previous similar messages [ 4815.411299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4816.162349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.134811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4817.920633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4818.854343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4819.635726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4820.358212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4821.166666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4822.190047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.053042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4823.876470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4824.736687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4825.596560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4826.580094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4827.464776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4828.447165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4829.286120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.075374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.889156] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4830.899234] Lustre: Skipped 18 previous similar messages [ 4830.917136] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4830.934322] Lustre: Skipped 18 previous similar messages [ 4830.938978] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4830.945805] Lustre: Skipped 18 previous similar messages [ 4830.949227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4830.963665] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4830.972667] Lustre: Skipped 18 previous similar messages [ 4831.888548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4832.772288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4833.863700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4834.815791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4835.807143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4836.652810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4837.525358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4838.451419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4839.339216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4840.193256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.100734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4841.828053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4842.669249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4843.505624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4844.632066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4845.446937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4846.190570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.128711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4847.995628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4848.779333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4849.680939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4850.643873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4851.406221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.294633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4852.966714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4853.798446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4854.718632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4855.607976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4856.518073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4857.599221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4858.684822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4859.510826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4860.482307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4861.561366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4862.487494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.460192] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4863.466959] Lustre: Skipped 35 previous similar messages [ 4863.479821] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4863.487371] Lustre: Skipped 35 previous similar messages [ 4863.490664] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4863.494837] Lustre: Skipped 35 previous similar messages [ 4863.498042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4863.516901] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4863.523809] Lustre: Skipped 35 previous similar messages [ 4864.237982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4865.264195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.073745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4866.829170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4867.851175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4868.718341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4869.629754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4870.650845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4871.250417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4872.150743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4873.056616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4874.255875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4875.024912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.008604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4876.999483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4877.846645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4878.519109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4879.404537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4880.241731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.043649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4881.793106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4882.564287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4883.341029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.072123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4884.752089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4885.568360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4886.338513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4887.222086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.094591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4888.892256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4889.672444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.469661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.168620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4891.857547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4892.572630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4893.329905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.027596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4894.872984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4895.769253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4896.507583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4897.166126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.005977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4898.804304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4899.651554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4900.556364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4901.348830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4902.107815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.052898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4903.920904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4904.762324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4905.540844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4906.594932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4907.514470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4908.287066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.149538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4909.902797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4910.623840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4911.388474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4912.384929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4913.290972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.109930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4914.868929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4915.626916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4916.565451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4917.378023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.167030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4918.985216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4919.829706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4920.650109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4921.411479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4922.275072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.068328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4923.881502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4924.773560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4925.705672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4926.409679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.187622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.909612] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4927.913898] Lustre: Skipped 77 previous similar messages [ 4927.927088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4927.936678] Lustre: Skipped 77 previous similar messages [ 4927.940414] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4927.945896] Lustre: Skipped 77 previous similar messages [ 4927.951647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4927.961580] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4927.974958] Lustre: Skipped 77 previous similar messages [ 4928.667833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4929.419442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4930.259180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.097835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4931.920359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4932.830564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4933.600689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4934.396653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4935.330720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.228159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.017039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4937.921226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4938.845129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4939.751132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4940.661204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4941.539125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4942.398787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.149610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4943.931221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4944.849537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4945.662710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.512738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4947.304585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.083882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4948.881791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4949.639771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4950.424324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4951.358047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4952.103462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4953.041423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4954.135059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4955.034481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.046591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4956.844437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4957.598252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4958.371226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4959.230674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.114433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4960.930466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4961.678896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4962.428596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.123477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4963.899829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4964.617608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4965.391062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.135193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4966.886225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4967.533636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.210413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4968.879155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4969.588097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.252595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4970.973750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4971.712658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4972.483782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4973.240550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.119735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4974.851253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4975.688588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4976.506651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.194940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4977.914852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4978.618773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4979.434328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.122107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4980.897956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4981.770880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4982.544951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4983.507162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4984.328045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4985.200741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4986.134245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.037650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4987.827364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4988.657309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4989.432910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4990.403573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4991.283486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.125941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4992.891601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4993.624487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4994.346543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4995.148185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.011621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4996.945414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4997.699898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4998.526823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4999.345538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.173591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.117912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.982199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.870391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5003.739832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.735357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.768674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.679958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.513870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.448867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.377655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5010.188461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.122797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.998899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.879108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.714335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.603313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.476617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.356357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.104986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5017.910606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.714551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5019.523857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.291943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.103462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.896929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.761203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.634867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.691035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5025.502182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.393961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.183239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.044752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.925426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.845268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.630033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.528555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5032.439485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.470659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.312258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.110960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.971689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5036.804049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5037.733333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.509298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.396657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.167754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.058651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.884711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.897701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.819085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.935893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.084508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.012274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.015946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.851563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.814367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.688694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.727291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.571939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.578487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.433733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.374723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.474425] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5056.488218] Lustre: Skipped 151 previous similar messages [ 5056.508551] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5056.526439] Lustre: Skipped 151 previous similar messages [ 5056.536016] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5056.541405] Lustre: Skipped 151 previous similar messages [ 5056.546789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.562802] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5056.578380] Lustre: Skipped 151 previous similar messages [ 5057.368564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.405869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.264199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.196206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.138927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.126245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.190629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.048755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.128050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.177308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.499401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5068.546776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.404533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.216254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.122895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.060309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.990794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.128483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.242567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.036087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.872537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.891171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.735878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.681560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.948990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.936419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.971468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.804393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.758275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.548085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.721311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.602054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5088.621732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.494376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.449507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.392478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.504508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.237600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.509213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.484593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.681325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.781041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.465584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.335258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.215392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.253902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.233354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.209420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.020162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.947782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.870123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.985459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.688178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.532398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.412030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.183431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.074605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.993420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.108922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.862839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.714816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.578902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.462191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.431265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.269262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.224385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.041508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.830501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.693407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.657321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.476140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.303758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.122429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.996728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.843196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.656692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.472075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.334610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.052402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.817936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.613519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.504138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.451731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.124336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.949537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.661751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.415511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.149445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.007156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.940484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.924284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.822536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.759290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.630126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.540555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.459626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.456024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.498720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.442089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.334081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.115821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.909637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.704838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.613730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.504891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.415280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.273289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.017808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.813401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.563812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.424556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.093710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.962992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.808232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.577499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.314299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.012626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.859884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.647470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.532861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.262872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.886867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.636487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.240353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.811229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.415970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.079854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.804761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.462626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.249057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.940462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.591774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.401775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.258903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.093768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.948278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.731177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.429985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.206148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.046032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.975560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.798788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.575244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.156954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.022374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.775034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.433626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.181435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.924112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.715858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.537767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.185926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.976799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.574159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.288563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.902979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.636745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.287249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.963301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.607709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.291905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.942482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.664661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.327322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.100633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.886268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.647604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.527298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.283724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.970889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.655115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.258736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.946490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.697055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.449213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.137799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.836741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.589899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.363411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.113538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.825730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.549045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.209677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.877924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.689773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.412646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.266379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.122768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.979083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.779720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.467035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.200516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.005442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.757832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.587317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.408532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.202939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.851778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.559708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.472428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.357752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.399193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.179928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.983600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.807018] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.723826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.472315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.096600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.919909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.901704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.615390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.333942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.991415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.715651] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5234.725564] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5234.737795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.560626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.369854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.130800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.915198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.849337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.525338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.231884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.995954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.721167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.556049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.314878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.038413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.851523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.774307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.737789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.706503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.725484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.628907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.407213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.297259] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5251.306483] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5251.310574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.525558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.829401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.026868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.639669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.966503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.934493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.031237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.045934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.350402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.754702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.115509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.294189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.683129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.333183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.349493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.238925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.240552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.214609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.998188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.110480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.302526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.289754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.393395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.322950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.250979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.041905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.827964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.691485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.685686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.677345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.823720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.777961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.708369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.453394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.477798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.436584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.465525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.282070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.094663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.677783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.286899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.870303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.761335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.600629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.510548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.307471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.031878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.855268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.597235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.332930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.191294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.881944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.686547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.560295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.429654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.130379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.974358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.738451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.580576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.350139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.128293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.808117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.579305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.424515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.216297] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5313.227244] Lustre: Skipped 298 previous similar messages [ 5313.240759] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5313.255286] Lustre: Skipped 298 previous similar messages [ 5313.260720] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5313.266704] Lustre: Skipped 298 previous similar messages [ 5313.270426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.276427] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5313.282213] Lustre: Skipped 298 previous similar messages [ 5313.954260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.731494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.372422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.999614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.752666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.496817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.343538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.080036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.781860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.555025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.328563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.121445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.848448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.633889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.527946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.287394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.933256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.649077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.522888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.309712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.033896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.815559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.586635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.301631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.002779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.797283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.722686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.732747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.539214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.434840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.272913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.079508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.926201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.903659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.797748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.729476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.562148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.415443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.328433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.257581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.951873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.761255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.543774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.316444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.126040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.836981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.562130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.302405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.045070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.879391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.806116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.702181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.474291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.313273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.037730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.923815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.681562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.544990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.385681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.236043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.095819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.056349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.945608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.909036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.716957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.461747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.346457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.300996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.267624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.258945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.227373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.180298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.944048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.765519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.480169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.369682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.146667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.900868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.768616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.711759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.502678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.415857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.442110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.516871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.426076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.493822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.667287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.731526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.600331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.658781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.485295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.172315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.883347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.668889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.459251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.488258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.150158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.006156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.023822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.945089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.992563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.794569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.870029] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5399.880084] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5399.889795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.860375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.680103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.509347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.447244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.252103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.115794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.936363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.865072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.625563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.433325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.258968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.897452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.711638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.436626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.203245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.933054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.788215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.577601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.308974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.133887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.021908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.819479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.678747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.352539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.095628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.941324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.799462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.664991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.594980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.426630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.350383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.159240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.029337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.830679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.690378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.499142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.365265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.181355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.068203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.836033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.597477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.268988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.990417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.672351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.468495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.113630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.901829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.570700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.353371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.027588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.830495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.508923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.364853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.080630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.888394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.661925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.535886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.299401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.072503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.784562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.578866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.356817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.351511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.280606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.356209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.004707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.956645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.885484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.943087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.027357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.114301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.890176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.864794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.848478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.875580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.865288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.952601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.679577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.683743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.377467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.451053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.297616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.068875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.846588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.846820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.755632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.581792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.495215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.564568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.289476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.079332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.982073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.946167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.689953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.516089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.390413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.075167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.068928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.819291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.610426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.410468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.372954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.245500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.145378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.972417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.736312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.630186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.572686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.377988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.264055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.105930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.842248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.718314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.572162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.544058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.397376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.213500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.988653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.800324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.620234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.580838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.372906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.333905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.242428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.157912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.002220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.804473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.789226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.739612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.782276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.818063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.845660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.687221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.588829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.658655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.690877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.614744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.651643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.671562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.475497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.330876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.033926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.791932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.606100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.394284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.174900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.814360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.721741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.626433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.392862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.340891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.207392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.042932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.906171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.745390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.553668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.312786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.092208] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5534.105057] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5534.115685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.893315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.630117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.429943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.239736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.145266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.040543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.803334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.649446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.677659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.671229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.412836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.311509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.237090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.161052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.912621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.768187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.584226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.521799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.436737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.395587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.252215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.172430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.960177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.877814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.747211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.656893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.647400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.567335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.425865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.281687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.085664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.908419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.698526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.700722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.814142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.754412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.718761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.585365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.460786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.466430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.419977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.448887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.445187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.426064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.484663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.362183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.135575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.946093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.854285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.686679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.617864] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5579.629807] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 5579.639059] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5579.643498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.645016] LustreError: 171844:0:(osp_precreate.c:1359:osp_precreate_thread()) Skipped 1 previous similar message [ 5580.510984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.395267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.251152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.159652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.020667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.837234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.670181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.385432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.125389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.984765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.777843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.733075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.542616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.667247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.448434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.282638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.163218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.966177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.894112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.842835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.856798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.905719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.819876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.758869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.863765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.806508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.834434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.815276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.749950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.814712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.892603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.036879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.060942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.111833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.226301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.235110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.073344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.936922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.055956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.065637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.106897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.078851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.111654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.117267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.970034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.040849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.061710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.071583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.907756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.710563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.665596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.575699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.450740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.325910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.119470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.066152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.886471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.679241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.359786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.214598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.093876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.048737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.930640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.917882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.762795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.691533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.575177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.427608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.288944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.170979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.021600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.921174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.977083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.969180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.824332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.694830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.447090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.222077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.023841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.818455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.588640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.499653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.251180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.982307] LustreError: 173645:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 5963.996616] LustreError: 171844:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6355.941491] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6355.943904] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6355.961468] Lustre: Skipped 396 previous similar messages [ 6360.437454] Lustre: server umount lustre-OST0000 complete [ 6364.641321] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6364.665125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6364.676743] Lustre: Skipped 1 previous similar message [ 6366.184808] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6366.197679] Lustre: Skipped 1 previous similar message [ 6370.480665] Lustre: server umount lustre-MDT0000 complete [ 6371.299683] LustreError: 172552:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6371.330716] LustreError: 172552:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6374.016326] LustreError: 171804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789602432 with bad export cookie 10738797858755370169 [ 6374.030269] LustreError: 171804:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6374.031484] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6374.375660] Lustre: server umount lustre-MDT0001 complete [ 6386.788078] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 6389.607362] Key type lgssc unregistered [ 6389.883531] LNet: 227017:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6389.888231] LNetError: 227017:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6389.913683] LNet: Removed LNI 192.168.206.152@tcp [ 6390.902365] Key type .llcrypt unregistered [ 6390.906337] Key type ._llcrypt unregistered [ 6413.537291] Key type ._llcrypt registered [ 6413.546121] Key type .llcrypt registered [ 6413.652810] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 6414.981122] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6415.004248] alg: No test for adler32 (adler32-zlib) [ 6416.166594] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 6416.713144] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 6418.463445] Key type lgssc registered [ 6419.983997] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6427.385393] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6427.402955] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6428.877431] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6433.277719] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6439.406737] LustreError: 228428:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6439.416172] LustreError: 228428:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6441.776425] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6442.209261] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6446.112749] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6454.449577] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6459.884574] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6466.019178] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6466.397828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6471.370592] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6474.634591] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 6477.862417] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6482.199965] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6483.652820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6483.668731] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6483.679845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.690951] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6483.696246] LustreError: 228448:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 6488.032905] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6488.044182] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6488.062947] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6489.688182] Lustre: lustre-OST0000: Not available for connect from 192.168.206.52@tcp (stopping) [ 6490.087537] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6494.799402] Lustre: lustre-OST0000: Not available for connect from 192.168.206.52@tcp (stopping) [ 6494.812521] Lustre: Skipped 2 previous similar messages [ 6499.298134] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6499.424834] Lustre: server umount lustre-OST0000 complete [ 6503.719926] LustreError: 228448:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 6533.652241] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6533.992082] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6540.740909] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6545.258053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.287446] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6547.798494] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6549.125418] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 6550.444245] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.458554] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 6554.500805] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f0944cd9800.ost_server_uuid 50 [ 6556.042151] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f0944cd9800.ost_server_uuid in FULL state after 0 sec [ 6560.736291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6560.749089] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6560.751340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6560.751347] Lustre: Skipped 2 previous similar messages [ 6560.792196] Lustre: Skipped 1 previous similar message [ 6562.249743] Lustre: server umount lustre-OST0000 complete [ 6565.857787] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6565.884777] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6565.891746] Lustre: Skipped 1 previous similar message [ 6570.466836] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6571.807982] Lustre: server umount lustre-MDT0000 complete [ 6575.391892] LustreError: 228410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789602633 with bad export cookie 7900878564713802694 [ 6575.398280] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6575.405362] LustreError: 228410:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 6575.694642] Lustre: server umount lustre-MDT0001 complete [ 6585.520045] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 6588.284658] Key type lgssc unregistered [ 6588.586749] LNet: 233633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6588.600692] LNetError: 233633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6588.613608] LNet: Removed LNI 192.168.206.152@tcp [ 6589.399539] Key type .llcrypt unregistered [ 6589.401974] Key type ._llcrypt unregistered [ 6620.400712] Key type ._llcrypt registered [ 6620.403318] Key type .llcrypt registered [ 6620.524875] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 6621.974537] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6622.015423] alg: No test for adler32 (adler32-zlib) [ 6623.027378] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 6623.201901] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 6624.848676] Key type lgssc registered [ 6626.067067] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6635.145858] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6635.176763] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6636.665171] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6641.387441] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6647.269853] LustreError: 235355:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6647.287024] LustreError: 235355:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6649.765636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6650.072360] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6654.241500] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6657.033525] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 6663.169892] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6663.503099] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6669.497292] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6677.313744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6677.533997] Lustre: lustre-OST0001: new disk, initializing [ 6677.539867] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 6677.546921] Lustre: 237661:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6677.624610] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6681.734457] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 6683.207167] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 6683.213600] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 6683.347498] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 6683.922925] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6694.651626] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6700.083907] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 6714.342768] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6714.345170] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6714.354304] Lustre: Skipped 2 previous similar messages [ 6714.363039] Lustre: Skipped 3 previous similar messages [ 6719.456634] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6719.468896] Lustre: Skipped 1 previous similar message [ 6720.454677] Lustre: server umount lustre-MDT0000 complete [ 6723.975602] LustreError: 235337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789602782 with bad export cookie 14469557696761746530 [ 6723.999386] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6724.002441] LustreError: 235337:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6724.345882] Lustre: server umount lustre-MDT0001 complete [ 6736.959240] Lustre: server umount lustre-OST0000 complete [ 6739.743232] Lustre: 234620:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789602782/real 1789602782] req@ffff8ea7c10af800 x1876534420931584/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789602798 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6739.766770] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6739.775823] Lustre: Skipped 1 previous similar message [ 6740.392940] Lustre: server umount lustre-OST0001 complete [ 6757.708649] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 6766.644269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6766.999822] LustreError: 240776:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6767.102770] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6771.252393] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6772.194517] LustreError: 240777:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6772.223385] LustreError: 240777:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 6777.318178] LustreError: 240776:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6779.902846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6780.387397] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6784.822638] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6787.819407] Lustre: 241917:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6794.844865] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6801.121404] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6806.511618] LustreError: 242269:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6806.541180] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 6809.398900] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6809.586087] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6809.591395] Lustre: Skipped 1 previous similar message [ 6812.676151] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 6816.444789] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6823.473726] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6826.716931] Lustre: 243787:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6836.707892] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6836.718964] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6836.730976] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6836.741046] Lustre: Skipped 2 previous similar messages [ 6842.875526] Lustre: server umount mylustre-MDT0000 complete [ 6843.369433] LustreError: 240776:0:(ldlm_lib.c:1199:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6843.406578] LustreError: 240776:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 6846.634866] LustreError: 240753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789602904 with bad export cookie 14469557696761749393 [ 6846.636020] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6846.654054] LustreError: 240753:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6847.147971] Lustre: server umount mylustre-MDT0001 complete [ 6861.319076] Lustre: server umount mylustre-OST0000 complete [ 6864.358296] Lustre: 234618:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789602906/real 1789602906] req@ffff8ea7e4f69880 x1876534420992384/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789602922 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6864.404378] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6864.422989] Lustre: Skipped 3 previous similar messages [ 6865.148826] Lustre: server umount mylustre-OST0001 complete [ 6885.649840] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 6895.361548] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6895.857527] LustreError: 246097:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6895.958929] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6900.859468] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6909.433669] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6914.733778] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6917.668188] Lustre: 247240:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 6923.937267] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6930.704605] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6935.524222] LustreError: 247594:0:(ldlm_lib.c:1199:target_handle_connect()) tfs-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6935.544573] LustreError: 247594:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 6935.562309] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 6939.241238] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6941.499556] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 6945.635558] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 6953.114194] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 6956.299084] Lustre: 249109:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 6962.148632] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6962.152314] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6962.153964] Lustre: Skipped 3 previous similar messages [ 6962.162953] Lustre: Skipped 7 previous similar messages [ 6967.276245] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 6967.282044] Lustre: Skipped 3 previous similar messages [ 6968.269150] Lustre: server umount tfs-MDT0000 complete [ 6971.619620] LustreError: 246841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603029 with bad export cookie 14469557696761751346 [ 6971.629219] LustreError: 246841:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 6971.642868] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6971.925706] Lustre: server umount tfs-MDT0001 complete [ 6984.902089] Lustre: server umount tfs-OST0000 complete [ 6988.257525] Lustre: 234620:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789603030/real 1789603030] req@ffff8ea7c359c000 x1876534421041536/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789603046 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 6988.284324] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6988.502879] Lustre: server umount tfs-OST0001 complete [ 7009.484435] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7019.369569] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7019.917626] LustreError: 251359:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7020.074538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7020.079488] Lustre: Skipped 3 previous similar messages [ 7024.659355] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7033.529955] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7038.397293] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7040.954533] Lustre: 252502:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7047.332039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7054.047211] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7059.094890] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7062.701646] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7065.025798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7069.238755] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7076.166773] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7079.109708] Lustre: 254375:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7085.881864] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:59:03 (1789603143) [ 7105.505675] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7105.513825] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7105.527142] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7107.937709] Lustre: server umount lustre-MDT0000 complete [ 7111.055343] LustreError: 252104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603169 with bad export cookie 14469557696761753033 [ 7111.059847] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7111.064841] LustreError: 252104:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 7117.496130] Lustre: server umount lustre-MDT0001 complete [ 7127.227328] Lustre: server umount lustre-OST0000 complete [ 7136.388695] Lustre: server umount lustre-OST0001 complete [ 7142.409382] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 7152.659547] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7186.308034] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7186.557925] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7186.582295] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7186.653125] Lustre: lustre-MDT0000: new disk, initializing [ 7186.724752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7186.734381] Lustre: Skipped 3 previous similar messages [ 7186.761386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7190.272793] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7199.663022] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7199.827198] Lustre: 257557:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7199.943455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7199.949656] Lustre: Skipped 1 previous similar message [ 7200.040916] Lustre: lustre-MDT0001: new disk, initializing [ 7200.201324] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7200.215303] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7204.753798] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7214.894813] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7215.248786] Lustre: lustre-OST0000: new disk, initializing [ 7215.258915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7215.271036] Lustre: 259151:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7216.471971] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7216.487953] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7216.591073] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7220.966832] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7231.255454] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7231.398754] Lustre: lustre-OST0001: new disk, initializing [ 7231.401651] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7231.407809] Lustre: 260022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7233.555934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7233.567900] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7233.698216] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7237.644536] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7249.309431] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 20:01:47 (1789603307) [ 7264.602640] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 20:02:02 (1789603322) [ 7279.592566] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7279.595418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7279.607362] Lustre: Skipped 6 previous similar messages [ 7279.613427] Lustre: Skipped 8 previous similar messages [ 7285.408349] Lustre: server umount lustre-OST0000 complete [ 7289.833537] LustreError: 260757:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7289.854064] LustreError: 260757:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 8 previous similar messages [ 7295.959533] Lustre: server umount lustre-MDT0000 complete [ 7299.757145] LustreError: 257552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603358 with bad export cookie 14469557696761754594 [ 7299.764080] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7299.766822] LustreError: 257552:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7314.400198] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7314.703388] Lustre: server umount lustre-MDT0001 complete [ 7325.867657] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 7398.846879] Lustre: server umount lustre-OST0001 complete [ 7405.539714] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 7424.058184] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7462.302129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7462.585368] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7462.633493] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7462.768632] Lustre: lustre-MDT0000: new disk, initializing [ 7462.887789] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7462.898426] Lustre: Skipped 3 previous similar messages [ 7462.919843] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7467.254876] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7477.680637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7477.853193] Lustre: 264882:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7478.220406] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7483.886752] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7493.573600] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7499.599215] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7506.431731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7506.793309] Lustre: lustre-OST0000: new disk, initializing [ 7506.799435] Lustre: Skipped 1 previous similar message [ 7506.803908] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7506.808794] Lustre: Skipped 2 previous similar messages [ 7506.814528] Lustre: 266847:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7508.667897] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7508.675018] Lustre: Skipped 1 previous similar message [ 7508.688456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7508.851727] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7513.724814] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7523.551513] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7534.561815] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7534.563368] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7534.570502] LustreError: Skipped 1 previous similar message [ 7534.572237] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7534.572244] Lustre: Skipped 13 previous similar messages [ 7534.609959] Lustre: Skipped 5 previous similar messages [ 7536.264206] Lustre: server umount lustre-OST0000 complete [ 7544.810299] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7546.507879] Lustre: server umount lustre-MDT0000 complete [ 7549.925300] LustreError: 264892:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7549.948059] LustreError: 264892:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 6 previous similar messages [ 7549.959393] LustreError: 264873:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603608 with bad export cookie 14469557696761756064 [ 7549.972962] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7550.320131] Lustre: server umount lustre-MDT0001 complete [ 7561.057633] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 7563.531555] Key type lgssc unregistered [ 7563.754831] LNet: 268607:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7563.760497] LNetError: 268607:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7564.774584] LNet: Removed LNI 192.168.206.152@tcp [ 7565.984219] Key type .llcrypt unregistered [ 7565.986463] Key type ._llcrypt unregistered [ 7593.467247] Key type ._llcrypt registered [ 7593.473823] Key type .llcrypt registered [ 7593.571666] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7594.644340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7594.669362] alg: No test for adler32 (adler32-zlib) [ 7595.671103] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 7595.909992] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 7597.575211] Key type lgssc registered [ 7599.138716] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7608.716757] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7608.736177] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7610.362285] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7615.084607] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7620.585182] LustreError: 270068:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7620.612868] LustreError: 270068:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 7624.106492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7624.595135] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7629.525651] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7636.550401] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7641.972406] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7648.767342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7649.103209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7655.289669] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 7655.608386] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7662.475596] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7663.320652] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 7665.645147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7665.649112] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7665.659515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7665.671443] Lustre: Skipped 1 previous similar message [ 7670.756166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7670.766429] Lustre: Skipped 1 previous similar message [ 7671.361218] Lustre: server umount lustre-OST0000 complete [ 7675.873296] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7675.880515] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7675.892073] LustreError: Skipped 1 previous similar message [ 7675.892676] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7675.913251] Lustre: Skipped 1 previous similar message [ 7675.934605] Lustre: Skipped 2 previous similar messages [ 7680.993522] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7681.005107] Lustre: Skipped 1 previous similar message [ 7681.465799] Lustre: server umount lustre-MDT0000 complete [ 7684.673344] LustreError: 270048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603743 with bad export cookie 1887451293273148804 [ 7684.674236] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7684.683828] LustreError: 270048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7684.959702] Lustre: server umount lustre-MDT0001 complete [ 7694.052214] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 7696.799228] Key type lgssc unregistered [ 7697.077652] LNet: 273649:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7697.086592] LNetError: 273649:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7697.101397] LNet: Removed LNI 192.168.206.152@tcp [ 7698.001080] Key type .llcrypt unregistered [ 7698.004067] Key type ._llcrypt unregistered [ 7713.930493] Key type ._llcrypt registered [ 7713.938395] Key type .llcrypt registered [ 7714.033576] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7715.265594] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7715.284800] alg: No test for adler32 (adler32-zlib) [ 7716.495783] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 7716.735373] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 7718.399276] Key type lgssc registered [ 7719.551905] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7727.797534] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7727.814666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7743.459695] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 7748.575536] LustreError: 274838:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.152@tcp: failed processing log, type 4: rc = -110 [ 7775.199480] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7781.803471] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 7789.183857] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7791.634468] Lustre: server umount lustre-OST0000 complete [ 7808.146405] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 7811.144801] Key type lgssc unregistered [ 7811.396980] LNet: 276468:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7811.410439] LNetError: 276468:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7811.440786] LNet: Removed LNI 192.168.206.152@tcp [ 7812.513358] Key type .llcrypt unregistered [ 7812.526953] Key type ._llcrypt unregistered [ 7854.751278] Key type ._llcrypt registered [ 7854.755345] Key type .llcrypt registered [ 7854.833619] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 7855.546754] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7855.568994] alg: No test for adler32 (adler32-zlib) [ 7856.782113] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 7857.082733] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 7858.743209] Key type lgssc registered [ 7860.000262] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7873.534671] loop0: detected capacity change from 0 to 204800000 [ 7873.764798] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7873.786632] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7874.130122] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7875.043339] loop0: detected capacity change from 0 to 204800000 [ 7875.270277] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7875.296460] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7875.821230] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7876.921806] loop0: detected capacity change from 0 to 204800000 [ 7877.091443] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7877.125740] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7877.196634] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7878.258100] loop0: detected capacity change from 0 to 204800000 [ 7878.435224] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 7879.561630] loop0: detected capacity change from 0 to 204800000 [ 7879.621429] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7884.264992] loop0: detected capacity change from 0 to 204800000 [ 7884.319568] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7888.608514] loop0: detected capacity change from 0 to 204800000 [ 7888.657755] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7892.936709] loop0: detected capacity change from 0 to 204800000 [ 7892.969177] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 7897.205177] loop0: detected capacity change from 0 to 204800000 [ 7897.254200] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7897.302107] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7897.373163] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7897.382070] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 7897.392083] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7897.402315] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7897.429030] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7897.445799] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 7897.458887] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 7897.469901] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 7897.483975] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 7897.491742] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 7897.504215] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 7897.522242] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 7897.534841] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 7897.546243] Lustre: 279879:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 7902.370270] Lustre: server umount MGS complete [ 7903.277623] loop0: detected capacity change from 0 to 204800000 [ 7903.390913] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7903.851956] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7903.898776] Lustre: lustre-MDT0000: Aborting client recovery [ 7904.775875] loop1: detected capacity change from 0 to 204800000 [ 7904.961124] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7905.055921] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7905.065716] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 7905.089624] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 7905.097578] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7905.105317] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7905.112483] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 7905.119494] Lustre: 280311:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7905.142436] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7905.273775] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7905.310287] Lustre: lustre-MDT0001: Aborting client recovery [ 7906.220719] loop2: detected capacity change from 0 to 204800000 [ 7906.365327] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7906.518474] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7906.558732] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7906.582092] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 7906.598606] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7906.612091] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 7906.642369] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 7906.661443] Lustre: 280391:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7906.773888] Lustre: Found index 0 for lustre-OST0000, updating log [ 7906.797385] Lustre: 280411:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7906.858803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7906.875852] Lustre: lustre-OST0000: Aborting recovery [ 7907.657456] loop3: detected capacity change from 0 to 204800000 [ 7907.753402] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7907.830603] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 7907.855923] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 7907.869144] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 7907.882915] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 7907.892799] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 7907.907758] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 7907.919384] Lustre: 280476:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 7907.970822] Lustre: Found index 1 for lustre-OST0001, updating log [ 7908.026485] Lustre: 280481:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7908.164055] Lustre: lustre-OST0001: Aborting recovery [ 7913.499583] LustreError: 280255:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 7914.466691] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7914.487345] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 7914.577626] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 7914.619406] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 7915.491142] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 7915.492752] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 7916.269072] LustreError: 280222:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 420us: rc = -2 [ 7916.773720] LustreError: 280712:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 733us: rc = -2 [ 7916.819987] LustreError: 280712:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 19 previous similar messages [ 7930.850538] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7930.855262] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7930.863960] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7930.870932] Lustre: Skipped 3 previous similar messages [ 7930.874724] Lustre: Skipped 2 previous similar messages [ 7935.571229] Lustre: server umount lustre-MDT0000 complete [ 7935.972995] LustreError: 280335:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7935.994976] LustreError: 280335:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [ 7936.533670] LustreError: 280204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789603994 with bad export cookie 2571454470179139787 [ 7936.539126] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7936.543335] LustreError: 280204:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7941.095473] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7941.096572] LustreError: 280712:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7941.130759] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7941.130768] Lustre: Skipped 1 previous similar message [ 7941.158859] LustreError: 280712:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 7943.262590] Lustre: server umount lustre-MDT0001 complete [ 7950.472086] Lustre: server umount lustre-OST0000 complete [ 7957.723467] Lustre: server umount lustre-OST0001 complete [ 7966.036912] Lustre: DEBUG MARKER: == conf-sanity test 108c: mkfs.lustre --mmp option enables MMP feature ========================================================== 20:13:43 (1789604023) [ 7986.627560] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 8028.198472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8028.514572] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8028.532256] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8028.584462] Lustre: lustre-MDT0000: new disk, initializing [ 8028.655541] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8028.671780] Lustre: Skipped 1 previous similar message [ 8028.695484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8033.242467] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8039.825729] Lustre: server umount lustre-MDT0000 complete [ 8056.012312] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:15:14 (1789604114) [ 8083.818101] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 8093.503367] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 8131.567358] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8131.828848] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8131.877431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8131.881346] Lustre: Skipped 1 previous similar message [ 8132.031879] Lustre: lustre-MDT0000: new disk, initializing [ 8132.106077] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8132.124168] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8136.381588] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8147.101494] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8147.250760] Lustre: 287576:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8147.312455] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8147.315467] Lustre: Skipped 1 previous similar message [ 8147.392893] Lustre: lustre-MDT0001: new disk, initializing [ 8147.507222] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8147.568488] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8147.589734] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8152.916481] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8163.039333] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8169.351560] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8176.345875] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8176.631395] Lustre: lustre-OST0000: new disk, initializing [ 8176.635492] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8176.646593] Lustre: 289546:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8176.751090] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8182.745492] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8185.392088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8185.400717] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8185.516964] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8192.961487] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8199.378240] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8208.398836] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8214.751269] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8252.386733] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8252.390605] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8252.401585] Lustre: Skipped 1 previous similar message [ 8252.411411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8252.420528] Lustre: Skipped 2 previous similar messages [ 8257.506282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8257.519346] Lustre: Skipped 1 previous similar message [ 8258.663078] Lustre: server umount lustre-OST0000 complete [ 8262.624601] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8262.638931] Lustre: Skipped 1 previous similar message [ 8262.644888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8262.653539] Lustre: Skipped 1 previous similar message [ 8268.745982] Lustre: server umount lustre-MDT0000 complete [ 8272.617152] LustreError: 287570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789604330 with bad export cookie 2571454470179142748 [ 8272.636752] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8272.638363] LustreError: 287570:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8273.079652] Lustre: server umount lustre-MDT0001 complete [ 8281.955025] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8286.740267] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8291.451557] Lustre: server umount MGS complete [ 8302.543564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8302.830444] LustreError: 293362:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8302.866651] LustreError: 293375:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8302.930854] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8307.940549] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8308.200691] LustreError: 293363:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8313.322075] LustreError: 293362:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8317.581254] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8322.927601] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8331.796899] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8337.916896] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8345.426370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8351.942741] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8359.804520] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8389.095211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8389.105810] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8389.117119] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8389.121499] Lustre: Skipped 2 previous similar messages [ 8389.140328] Lustre: Skipped 3 previous similar messages [ 8394.549355] Lustre: server umount lustre-OST0000 complete [ 8399.330881] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8399.347765] Lustre: Skipped 1 previous similar message [ 8403.871184] Lustre: server umount lustre-MDT0000 complete [ 8404.452995] LustreError: 293357:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8404.482906] LustreError: 293357:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8408.934429] LustreError: 293343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789604467 with bad export cookie 2571454470179144603 [ 8408.945519] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8408.955022] LustreError: 293343:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8409.402947] Lustre: server umount lustre-MDT0001 complete [ 8421.213665] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 8424.407696] Key type lgssc unregistered [ 8424.633624] LNet: 297308:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8424.642762] LNetError: 297308:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8425.704035] LNet: Removed LNI 192.168.206.152@tcp [ 8426.763253] Key type .llcrypt unregistered [ 8426.775374] Key type ._llcrypt unregistered [ 8463.682571] Key type ._llcrypt registered [ 8463.687216] Key type .llcrypt registered [ 8463.812567] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 8480.913587] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 8482.161615] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8482.321759] alg: No test for adler32 (adler32-zlib) [ 8483.721271] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 8484.030624] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 8485.727240] Key type lgssc registered [ 8486.949054] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8523.608823] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8523.672551] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8524.991986] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8525.049726] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8525.186337] Lustre: lustre-MDT0000: new disk, initializing [ 8525.294562] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8525.308718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8528.871486] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8539.347155] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8539.501371] Lustre: 301262:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8539.568697] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8539.575757] Lustre: Skipped 1 previous similar message [ 8539.642291] Lustre: lustre-MDT0001: new disk, initializing [ 8539.740813] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8539.777255] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8539.785474] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8544.127559] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8554.529647] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8560.747212] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8567.862234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8568.267194] Lustre: lustre-OST0000: new disk, initializing [ 8568.270264] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8568.274910] Lustre: 303230:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8568.349817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8575.310147] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8576.624272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8576.643211] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8576.792598] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8585.523556] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8592.205781] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8604.659973] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8608.670485] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8616.980195] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8664.032662] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8664.044967] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8664.072476] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8664.081231] Lustre: Skipped 1 previous similar message [ 8666.567027] Lustre: server umount lustre-OST0000 complete [ 8672.738941] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8672.765950] LustreError: Skipped 1 previous similar message [ 8672.770103] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8672.796402] Lustre: Skipped 1 previous similar message [ 8672.804567] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8674.275917] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8674.281576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8674.294193] Lustre: Skipped 1 previous similar message [ 8677.525535] Lustre: server umount lustre-MDT0000 complete [ 8682.657295] LustreError: 303229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789604741 with bad export cookie 13227551173937015785 [ 8682.663428] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8682.671575] LustreError: 303229:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8683.064731] Lustre: server umount lustre-MDT0001 complete [ 8692.728828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8697.947627] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8703.026046] Lustre: server umount MGS complete [ 8715.225367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8715.868014] LustreError: 307097:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8715.891764] LustreError: 307097:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 8715.920493] LustreError: 307110:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8715.996620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8721.069841] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8721.383977] LustreError: 307098:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8726.499192] LustreError: 307097:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8728.912995] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8729.302176] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8734.191074] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8741.826452] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8747.144486] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8753.925870] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8754.242199] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8760.567444] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8768.533983] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8795.104409] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8795.123518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8795.142390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8795.693958] Lustre: server umount lustre-OST0000 complete [ 8801.249108] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8801.267907] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8801.297717] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8810.525925] Lustre: server umount lustre-MDT0000 complete [ 8812.515744] LustreError: 309545:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8812.541382] LustreError: 309545:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 8814.074291] LustreError: 309940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789604872 with bad export cookie 13227551173937017626 [ 8814.076946] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8814.088387] LustreError: 309940:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8814.405218] Lustre: server umount lustre-MDT0001 complete [ 8826.280367] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 8829.209464] Key type lgssc unregistered [ 8829.492662] LNet: 311043:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8829.504424] LNetError: 311043:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8829.516818] LNet: Removed LNI 192.168.206.152@tcp [ 8830.388381] Key type .llcrypt unregistered [ 8830.391571] Key type ._llcrypt unregistered [ 8852.084475] Key type ._llcrypt registered [ 8852.086622] Key type .llcrypt registered [ 8852.175962] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 8865.800750] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 8866.979658] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8867.121104] alg: No test for adler32 (adler32-zlib) [ 8868.412534] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [ 8868.776081] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [ 8870.487240] Key type lgssc registered [ 8871.828826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8895.787911] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 8908.097388] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8908.135897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8909.285063] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8909.301543] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8909.347136] Lustre: lustre-MDT0000: new disk, initializing [ 8909.415114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8909.429914] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8913.013507] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8922.395700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8922.741551] Lustre: lustre-OST0000: new disk, initializing [ 8922.746851] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8922.754164] Lustre: Skipped 1 previous similar message [ 8922.761373] Lustre: 315089:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8922.867382] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8928.032419] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 8929.384365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8929.392672] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8929.423930] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9676.769336] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9676.790472] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9681.025527] Lustre: server umount lustre-MDT0000 complete [ 9684.923988] LustreError: 314275:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789605743 with bad export cookie 7089308688179256004 [ 9684.926014] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9684.941733] LustreError: 314275:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9685.119309] Lustre: server umount lustre-OST0000 complete [ 9709.593916] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [ 9718.018216] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [ 9746.618214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9746.785109] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9746.807822] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9746.879702] Lustre: lustre-MDT0000: new disk, initializing [ 9746.931266] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9746.944667] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9750.153273] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9758.621933] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9758.751607] Lustre: 319410:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9758.798874] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9758.803695] Lustre: Skipped 1 previous similar message [ 9758.911561] Lustre: lustre-MDT0001: new disk, initializing [ 9759.034516] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9759.082039] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9759.099609] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9763.072648] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9772.068692] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9776.885867] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9783.263895] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9783.469810] Lustre: lustre-OST0000: new disk, initializing [ 9783.472917] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9783.477883] Lustre: 321374:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9783.539142] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9789.167867] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9790.076299] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9790.092556] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9790.170618] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9798.257726] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9803.450649] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9803.692936] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9808.744547] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9808.972971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9816.044818] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9816.045124] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9816.047554] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9816.057634] Lustre: Skipped 1 previous similar message [ 9817.285678] Lustre: server umount lustre-OST0000 complete [ 9820.646364] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9820.653668] LustreError: Skipped 1 previous similar message [ 9820.657436] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9820.668379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9820.676977] Lustre: Skipped 1 previous similar message [ 9826.275929] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9826.286040] Lustre: Skipped 3 previous similar messages [ 9826.815411] Lustre: server umount lustre-MDT0000 complete [ 9829.776358] LustreError: 319401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789605888 with bad export cookie 7089308688182196823 [ 9829.789768] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9829.798492] LustreError: 319401:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9829.981816] Lustre: server umount lustre-MDT0001 complete [ 9832.413949] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 9833.896608] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:44:52 (1789605892) [ 9839.918485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9840.420604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9844.256666] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9845.728980] LustreError: 323689:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9845.744252] LustreError: 323689:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [ 9849.829177] LustreError: 323688:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9851.548344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9851.943571] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9855.866381] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9862.762496] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9868.144596] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9874.766744] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9875.194904] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9881.362812] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9887.399740] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9893.483907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9893.677843] Lustre: lustre-OST0001: new disk, initializing [ 9893.682551] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9893.691377] Lustre: 326523:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9893.811920] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9899.582850] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [ 9900.683615] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9900.697822] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9900.739257] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9901.800090] LustreError: 326946:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9901.809798] LustreError: 326946:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [ 9902.821410] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9904.873665] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9904.888254] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9907.823672] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9908.030433] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9909.030257] LustreError: 326946:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9909.053745] LustreError: 326946:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [ 9912.613778] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9f09464c2000.ost_server_uuid 50 [ 9914.092396] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9f09464c2000.ost_server_uuid in FULL state after 0 sec [ 9917.222025] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9917.231261] LustreError: 326955:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [ 9919.384397] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9f09464c2000.ost_server_uuid 50 [ 9920.748955] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9f09464c2000.ost_server_uuid in FULL state after 0 sec [ 9922.426415] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9934.944162] Lustre: lustre-OST0001: Not available for connect from 192.168.206.52@tcp (stopping) [ 9936.879523] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9936.888205] Lustre: Skipped 2 previous similar messages [ 9945.160493] Lustre: lustre-OST0001: Not available for connect from 192.168.206.52@tcp (stopping) [ 9945.167639] Lustre: Skipped 5 previous similar messages [ 9949.151285] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9949.338905] Lustre: server umount lustre-OST0001 complete [ 9950.275972] LustreError: 325510:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.206.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9954.278939] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9954.286518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9957.344262] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9957.360204] LustreError: 325512:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9957.372770] LustreError: 325512:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [ 9960.300522] Lustre: server umount lustre-OST0000 complete [ 9964.516820] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9964.524634] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9964.531104] Lustre: Skipped 1 previous similar message [ 9964.534374] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9964.541062] Lustre: Skipped 5 previous similar messages [ 9977.311791] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9977.524314] Lustre: server umount lustre-MDT0000 complete [ 9977.824157] LustreError: 323683:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9977.840623] LustreError: 323683:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [ 9981.494649] LustreError: 323669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789606039 with bad export cookie 7089308688182197705 [ 9981.498870] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9981.501373] LustreError: 323669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9981.723910] Lustre: server umount lustre-MDT0001 complete [ 9992.355868] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [ 9994.427862] Key type lgssc unregistered [ 9994.658465] LNet: 329211:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9994.664994] LNetError: 329211:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9994.679766] LNet: Removed LNI 192.168.206.152@tcp [ 9995.225725] Key type .llcrypt unregistered [ 9995.227594] Key type ._llcrypt unregistered [10018.922461] Key type ._llcrypt registered [10018.924639] Key type .llcrypt registered [10019.049366] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10020.161320] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10020.201689] alg: No test for adler32 (adler32-zlib) [10021.307964] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [10021.545862] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [10023.199492] Key type lgssc registered [10024.356707] Lustre: Echo OBD driver; http://www.lustre.org/ [10031.670657] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10031.684131] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10033.089826] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10037.606991] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10043.362506] LustreError: 330637:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10043.405357] LustreError: 330637:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10045.297982] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10048.509066] LustreError: 330636:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10051.692410] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10052.011992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10056.342235] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10064.990009] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10065.281769] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10071.214891] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10077.546717] LustreError: 332317:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10078.256754] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10078.576894] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10083.817653] LustreError: 332319:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10083.829029] LustreError: 332319:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10084.435657] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10084.770464] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10089.983077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10090.665627] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10098.318740] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10105.107458] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10105.341778] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10207.720858] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10207.728679] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10207.753747] LustreError: Skipped 1 previous similar message [10207.754281] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10207.771549] Lustre: Skipped 1 previous similar message [10209.934126] Lustre: server umount lustre-OST0001 complete [10212.832421] LustreError: 332319:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10212.865435] LustreError: 332319:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10215.904513] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10215.912272] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10215.933565] Lustre: Skipped 1 previous similar message [10215.943200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10216.929218] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10216.941050] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10216.961634] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10221.559646] Lustre: server umount lustre-OST0000 complete [10226.145107] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10226.152308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10226.173608] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10226.182991] Lustre: Skipped 2 previous similar messages [10231.824502] Lustre: server umount lustre-MDT0000 complete [10233.312891] LustreError: 330637:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10233.340374] LustreError: 330637:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10235.886940] LustreError: 334319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789606294 with bad export cookie 79171741986495642 [10235.889206] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10235.914569] LustreError: 334319:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10236.276590] Lustre: server umount lustre-MDT0001 complete [10247.123353] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [10250.236736] Key type lgssc unregistered [10250.554776] LNet: 335577:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10250.562164] LNetError: 335577:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10250.578737] LNet: Removed LNI 192.168.206.152@tcp [10251.444590] Key type .llcrypt unregistered [10251.451497] Key type ._llcrypt unregistered [10295.480768] Key type ._llcrypt registered [10295.483835] Key type .llcrypt registered [10295.599004] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10296.421225] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10296.444047] alg: No test for adler32 (adler32-zlib) [10297.503638] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [10297.686970] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [10299.513794] Key type lgssc registered [10301.253946] Lustre: Echo OBD driver; http://www.lustre.org/ [10311.484422] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10311.517823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10313.080182] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10317.510900] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10323.426087] LustreError: 337860:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10323.462793] LustreError: 337860:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [10325.731488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10326.054675] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10330.962703] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10333.918883] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10340.184912] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10340.568773] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10346.134685] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10350.836465] LustreError: 339352:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10355.480928] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10355.759706] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10357.816871] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [10357.835651] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [10363.140463] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10373.232677] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10384.231227] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10397.664834] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10397.677715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10397.716112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10398.699723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10398.705883] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10398.719516] Lustre: Skipped 2 previous similar messages [10398.746811] Lustre: Skipped 2 previous similar messages [10402.405237] Lustre: server umount lustre-MDT0000 complete [10403.817581] LustreError: 337856:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10403.859241] LustreError: 337856:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10406.563838] LustreError: 337842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789606464 with bad export cookie 10882640864219172177 [10406.566991] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10406.581894] LustreError: 337842:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10406.958035] Lustre: server umount lustre-MDT0001 complete [10421.640226] Lustre: server umount lustre-OST0000 complete [10435.674867] Lustre: server umount lustre-OST0001 complete [10453.369481] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10464.172789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10464.659883] LustreError: 343128:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10464.758847] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10469.042121] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10469.857333] LustreError: 343129:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10477.696696] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10478.195714] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10482.554977] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10485.370858] Lustre: 344267:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10492.120385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10499.350720] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10502.783374] LustreError: 344621:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10502.810925] LustreError: 344621:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10507.642504] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10507.947850] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10507.953439] Lustre: Skipped 1 previous similar message [10510.014882] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [10510.029706] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [10513.645232] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10521.284601] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10525.000629] Lustre: 346140:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10529.254037] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10529.264862] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10529.292917] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10529.298145] Lustre: Skipped 1 previous similar message [10535.100786] Lustre: server umount lustre-MDT0000 complete [10535.905157] LustreError: 344383:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10535.920025] LustreError: 344383:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 3 previous similar messages [10539.143476] LustreError: 346142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789606597 with bad export cookie 10882640864219173878 [10539.144995] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10539.157253] LustreError: 346142:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10539.540548] Lustre: server umount lustre-MDT0001 complete [10552.825054] Lustre: server umount lustre-OST0000 complete [10566.547174] Lustre: server umount lustre-OST0001 complete [10570.148164] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [10571.926452] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [10573.622424] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:57:11 (1789606631) [10596.837780] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10605.635049] loop0: detected capacity change from 0 to 1099511627776 [10606.338682] SGI XFS with ACLs, security attributes, quota, no debug enabled [10606.347360] *** VALIDATE xfs *** [10606.382603] XFS (loop0): Mounting V5 Filesystem [10606.445737] XFS (loop0): Ending clean mount [10607.542920] loop1: detected capacity change from 0 to 18691697672192 [10721.559551] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [10731.663317] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10771.569764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10771.830552] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10771.866513] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10771.942559] Lustre: lustre-MDT0000: new disk, initializing [10772.046575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10772.063963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10776.664408] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10787.204056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10787.360924] Lustre: 351281:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10787.419846] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10787.426269] Lustre: Skipped 1 previous similar message [10787.498226] Lustre: lustre-MDT0001: new disk, initializing [10787.629912] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10787.636716] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10791.468751] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10800.150973] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10805.885264] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10812.585430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10812.959661] Lustre: lustre-OST0000: new disk, initializing [10812.962741] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10812.968279] Lustre: 353260:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10814.714600] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10814.720349] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10814.834833] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10818.684823] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10826.708477] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10831.103722] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10831.334183] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10835.468400] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10835.619814] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10840.544502] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10840.561494] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10840.574354] Lustre: Skipped 3 previous similar messages [10840.580854] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10840.589633] Lustre: Skipped 4 previous similar messages [10845.664950] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10845.674623] Lustre: Skipped 1 previous similar message [10851.295103] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10851.414149] Lustre: server umount lustre-OST0000 complete [10854.368235] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10854.373954] LustreError: Skipped 1 previous similar message [10854.377452] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10854.386073] Lustre: Skipped 1 previous similar message [10854.392206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10854.395807] Lustre: Skipped 3 previous similar messages [10860.585717] Lustre: server umount lustre-MDT0000 complete [10861.026912] LustreError: 351286:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10861.054730] LustreError: 351286:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [10863.916352] LustreError: 353259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789606922 with bad export cookie 10882640864219175432 [10863.923567] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10863.923679] LustreError: 353259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10864.105885] Lustre: server umount lustre-MDT0001 complete [10870.801516] XFS (loop0): Unmounting Filesystem [10874.990177] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 21:02:13 (1789606933) [10882.935250] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10883.741439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10883.749966] Lustre: Skipped 2 previous similar messages [10889.028924] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10897.283966] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10901.894114] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10908.075857] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10912.910408] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10919.242535] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10924.479920] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [10930.338865] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10938.336278] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10938.347923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10938.356866] Lustre: Skipped 1 previous similar message [10938.366420] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10938.371850] Lustre: Skipped 2 previous similar messages [10943.781044] Lustre: server umount lustre-OST0000 complete [10949.089942] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10949.110155] LustreError: Skipped 1 previous similar message [10953.799463] Lustre: server umount lustre-MDT0000 complete [10955.764080] LustreError: 355819:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10955.801439] LustreError: 355819:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [10957.524756] LustreError: 355800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789607015 with bad export cookie 10882640864219176314 [10957.529154] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10957.541816] LustreError: 355800:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10957.947923] Lustre: server umount lustre-MDT0001 complete [10968.681798] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [10970.768292] Key type lgssc unregistered [10970.981794] LNet: 359446:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10970.987437] LNetError: 359446:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10970.998361] LNet: Removed LNI 192.168.206.152@tcp [10971.604170] Key type .llcrypt unregistered [10971.605958] Key type ._llcrypt unregistered [10995.584076] Key type ._llcrypt registered [10995.586169] Key type .llcrypt registered [10995.737423] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [10996.945579] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10996.993607] alg: No test for adler32 (adler32-zlib) [10998.156991] Lustre: Lustre: Build Version: 2.17.58_39_g6aa67ea [10998.412557] LNet: Added LNI 192.168.206.152@tcp [8/256/0/180] [11000.095192] Key type lgssc registered [11001.150881] Lustre: Echo OBD driver; http://www.lustre.org/ [11009.032189] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11009.050933] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11010.567219] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11015.041279] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11019.759853] LustreError: 361009:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11019.784991] LustreError: 361009:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11023.163685] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11023.592910] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11028.095766] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11035.184882] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11040.542099] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11045.790969] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11046.127789] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11051.224036] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11057.257986] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11064.802926] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11064.817795] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11064.834493] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11065.827247] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11065.827892] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11069.035641] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.52@tcp (stopping) [11069.040875] Lustre: Skipped 1 previous similar message [11070.776384] Lustre: server umount lustre-MDT0001 complete [11070.945803] LustreError: 361010:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11070.990268] LustreError: 361010:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 1 previous similar message [11074.117414] LustreError: 361004:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11079.244765] LustreError: 361005:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11079.278524] LustreError: 361005:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 2 previous similar messages [11080.978926] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11081.067553] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11081.069900] Lustre: Found index 1 for lustre-MDT0001, updating log [11081.093644] Lustre: 360997:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11081.185838] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [11081.189379] Lustre: Skipped 1 previous similar message [11081.312613] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11084.420445] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11086.326339] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11086.373800] LustreError: 364585:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11086.381365] LustreError: 364585:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11086.401790] LustreError: 361001:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11086.476543] LustreError: 364590:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.152@tcp: cfg command failed: rc = -17 [11086.499100] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11086.499100] [11094.627707] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.52@tcp (stopping) [11096.545982] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11096.552290] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11098.800410] Lustre: server umount lustre-MDT0001 complete [11099.711264] LustreError: 363499:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.52@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11108.608557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11108.741331] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11108.748671] Lustre: Found index 1 for lustre-MDT0001, updating log [11108.802565] Lustre: 360999:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11109.129879] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11113.548563] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11114.481145] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11114.507247] LustreError: 365582:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11114.525744] LustreError: 365582:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11114.544329] LustreError: 361001:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11114.553880] LustreError: 361001:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11114.639240] LustreError: 365593:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [11114.643868] LustreError: 365593:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [11114.648961] LustreError: 365593:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.152@tcp: cfg command failed: rc = -17 [11114.660120] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.152@tcp [11114.660120] [11421.665101] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11421.669169] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11421.687670] Lustre: Skipped 1 previous similar message [11421.690502] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11421.690509] Lustre: Skipped 2 previous similar messages [11424.342345] Lustre: server umount lustre-MDT0001 complete [11426.796317] LustreError: 361738:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11426.812787] LustreError: 361738:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 5 previous similar messages [11434.410980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11434.478790] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11434.483178] Lustre: Found index 1 for lustre-MDT0001, updating log [11434.540230] Lustre: 360998:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11434.851515] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [11438.623545] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11440.116160] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11440.130777] LustreError: 374760:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11440.136891] LustreError: 374760:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11440.150797] LustreError: 361001:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11440.155207] LustreError: 361001:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [11440.252579] LustreError: 374771:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [11440.272348] LustreError: 374771:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.152@tcp: cfg command failed: rc = -114 [11440.286088] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [11440.286088] [11762.667351] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11762.669051] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11762.672340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11762.672348] Lustre: Skipped 2 previous similar messages [11762.679132] Lustre: Skipped 2 previous similar messages [11765.578610] Lustre: server umount lustre-MDT0000 complete [11767.785287] LustreError: 361004:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11767.808779] LustreError: 361004:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 7 previous similar messages [11768.991779] LustreError: 362852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789607827 with bad export cookie 1784444458577535783 [11768.995419] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11769.009407] LustreError: 362852:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11772.905682] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11772.923991] Lustre: Skipped 1 previous similar message [11775.395560] Lustre: server umount lustre-MDT0001 complete [11778.948531] Lustre: server umount lustre-OST0000 complete [11788.383222] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [11797.789554] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [11832.246280] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:18:10 (1789607890) [11838.672738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11838.967854] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11838.999980] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11839.078441] Lustre: lustre-MDT0000: new disk, initializing [11839.162089] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11839.185575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11843.531636] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11853.421883] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11853.529805] Lustre: 386185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11853.588733] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11853.592306] Lustre: Skipped 1 previous similar message [11853.697644] Lustre: lustre-MDT0001: new disk, initializing [11853.814663] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11853.855444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11853.868539] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11858.066774] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11865.816474] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11870.048342] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11875.032809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11875.230944] Lustre: lustre-OST0000: new disk, initializing [11875.235421] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11875.245494] Lustre: 388152:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11875.348835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11881.817774] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_default_debug -1 all [11883.582283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11883.599410] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11883.749514] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11890.091792] Lustre: DEBUG MARKER: oleg652-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11898.861265] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [11898.875659] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11898.890913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11898.897121] Lustre: Skipped 4 previous similar messages [11903.041848] Lustre: server umount lustre-MDT0000 complete [11904.482127] LustreError: 386198:0:(ldlm_lib.c:1199:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11904.496017] LustreError: 386198:0:(ldlm_lib.c:1199:target_handle_connect()) Skipped 4 previous similar messages [11906.791430] LustreError: 386177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789607965 with bad export cookie 1784444458577537736 [11906.799317] LustreError: MGC192.168.206.152@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11906.806378] LustreError: 386177:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11906.830198] LustreError: 386992:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [11907.186449] Lustre: server umount lustre-MDT0001 complete [11921.514307] Lustre: server umount lustre-OST0000 complete [11948.055626] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing set_hostid [11959.049212] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing load_modules_local [11991.113557] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 11698 sec ========= 21:20:48 (1789608048) [11993.107608] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:20:50 (1789608050) === [11997.081857] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:20:54 (1789608054) === [12039.408976] Lustre: DEBUG MARKER: oleg652-server.virtnet: executing unload_modules_local [12042.131582] Key type lgssc unregistered [12042.468614] LNet: 394520:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12042.476184] LNetError: 394520:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12042.496038] LNet: Removed LNI 192.168.206.152@tcp [12043.400521] Key type .llcrypt unregistered [12043.402935] Key type ._llcrypt unregistered