[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 680818591 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524588K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003121] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005018] kvm-guest: setup PV IPIs [ 0.008742] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.010024] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.011020] pid_max: default: 32768 minimum: 301 [ 0.012146] LSM: Security Framework initializing [ 0.014137] Yama: becoming mindful. [ 0.015042] SELinux: Initializing. [ 0.016061] *** VALIDATE selinux *** [ 0.026030] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.031850] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.032200] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033131] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.034141] *** VALIDATE tmpfs *** [ 0.036061] *** VALIDATE proc *** [ 0.037378] *** VALIDATE cgroup *** [ 0.039008] *** VALIDATE cgroup2 *** [ 0.040403] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041258] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042014] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043034] Spectre V2 : User space: Vulnerable [ 0.044009] Speculative Store Bypass: Vulnerable [ 0.049112] debug: unmapping init [mem 0xffffffffa3c59000-0xffffffffa3c60fff] [ 0.052000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.053152] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.054175] ... version: 2 [ 0.055022] ... bit width: 48 [ 0.056021] ... generic registers: 4 [ 0.057021] ... value mask: 0000ffffffffffff [ 0.058021] ... max period: 00007fffffffffff [ 0.059023] ... fixed-purpose events: 3 [ 0.060084] ... event mask: 000000070000000f [ 0.061369] rcu: Hierarchical SRCU implementation. [ 0.064058] smp: Bringing up secondary CPUs ... [ 0.066105] x86: Booting SMP configuration: [ 0.067021] .... node #0, CPUs: #1 #2 #3 [ 0.079216] smp: Brought up 1 node, 4 CPUs [ 0.081143] smpboot: Max logical packages: 1 [ 0.082018] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.144686] node 0 deferred pages initialised in 60ms [ 0.152487] devtmpfs: initialized [ 0.153625] x86/mm: Memory block size: 128MB [ 0.158013] gcov: version magic: 0x41383552 [ 0.162611] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.163110] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.165204] pinctrl core: initialized pinctrl subsystem [ 0.166649] [ 0.167015] ************************************************************* [ 0.168027] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.169023] ** ** [ 0.170022] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.171018] ** ** [ 0.172017] ** This means that this kernel is built to expose internal ** [ 0.173024] ** IOMMU data structures, which may compromise security on ** [ 0.174026] ** your system. ** [ 0.175021] ** ** [ 0.176027] ** If you see this message and you are not debugging the ** [ 0.177029] ** kernel, report this immediately to your vendor! ** [ 0.178022] ** ** [ 0.179029] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.180021] ************************************************************* [ 0.182649] NET: Registered protocol family 16 [ 0.183663] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.184096] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.185108] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.187640] cpuidle: using governor menu [ 0.191814] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.198387] PCI: Using configuration type 1 for base access [ 0.202154] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.216103] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.218027] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.224182] cryptd: max_cpu_qlen set to 1000 [ 0.226617] ACPI: Added _OSI(Module Device) [ 0.227028] ACPI: Added _OSI(Processor Device) [ 0.230020] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.233023] ACPI: Added _OSI(Processor Aggregator Device) [ 0.245000] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.255980] ACPI: Interpreter enabled [ 0.256096] ACPI: PM: (supports S0 S3 S4 S5) [ 0.257016] ACPI: Using IOAPIC for interrupt routing [ 0.258152] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.259467] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.271888] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.272123] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.273023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.274112] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.276883] acpiphp: Slot [2] registered [ 0.277255] acpiphp: Slot [5] registered [ 0.278232] acpiphp: Slot [6] registered [ 0.279398] acpiphp: Slot [7] registered [ 0.280271] acpiphp: Slot [8] registered [ 0.281140] acpiphp: Slot [9] registered [ 0.282250] acpiphp: Slot [10] registered [ 0.283207] acpiphp: Slot [3] registered [ 0.284171] acpiphp: Slot [4] registered [ 0.285148] acpiphp: Slot [11] registered [ 0.286160] acpiphp: Slot [12] registered [ 0.287128] acpiphp: Slot [13] registered [ 0.288293] acpiphp: Slot [14] registered [ 0.289141] acpiphp: Slot [15] registered [ 0.290165] acpiphp: Slot [16] registered [ 0.291183] acpiphp: Slot [17] registered [ 0.292159] acpiphp: Slot [18] registered [ 0.293145] acpiphp: Slot [19] registered [ 0.294154] acpiphp: Slot [20] registered [ 0.295190] acpiphp: Slot [21] registered [ 0.296182] acpiphp: Slot [22] registered [ 0.297145] acpiphp: Slot [23] registered [ 0.298435] acpiphp: Slot [24] registered [ 0.299204] acpiphp: Slot [25] registered [ 0.300149] acpiphp: Slot [26] registered [ 0.301181] acpiphp: Slot [27] registered [ 0.302143] acpiphp: Slot [28] registered [ 0.303164] acpiphp: Slot [29] registered [ 0.304130] acpiphp: Slot [30] registered [ 0.305209] acpiphp: Slot [31] registered [ 0.306082] PCI host bridge to bus 0000:00 [ 0.307028] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.308078] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.309037] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.310042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.311033] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.312033] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.313214] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.315344] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.317977] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.324024] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.328040] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.329025] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.330027] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.331027] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.333309] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.335069] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.336051] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.338973] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.341025] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.348042] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.351026] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.359412] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.362028] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.365022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.372024] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.386791] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.413033] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.433023] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.474026] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.497000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.518020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.539024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.581023] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.613262] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.646019] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.658019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.695024] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.717000] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.731020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.742020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.769020] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.785018] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.810027] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.823039] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.868026] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.893533] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.899416] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.904450] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.911998] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.918014] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.929053] iommu: Default domain type: Passthrough [ 0.934256] SCSI subsystem initialized [ 0.935180] ACPI: bus type USB registered [ 0.937293] usbcore: registered new interface driver usbfs [ 0.941151] usbcore: registered new interface driver hub [ 0.943142] usbcore: registered new device driver usb [ 0.945417] pps_core: LinuxPPS API ver. 1 registered [ 0.948032] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.953141] PTP clock support registered [ 0.958132] EDAC MC: Ver: 3.0.0 [ 0.959015] PCI: Using ACPI for IRQ routing [ 0.961436] NetLabel: Initializing [ 0.963019] NetLabel: domain hash size = 128 [ 0.966014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.970108] NetLabel: unlabeled traffic allowed by default [ 0.975374] vgaarb: loaded [ 0.977637] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.980017] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.990583] clocksource: Switched to clocksource kvm-clock [ 1.214114] VFS: Disk quotas dquot_6.6.0 [ 1.215831] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.220116] *** VALIDATE ramfs *** [ 1.227572] *** VALIDATE hugetlbfs *** [ 1.232945] pnp: PnP ACPI init [ 1.244881] pnp: PnP ACPI: found 6 devices [ 1.273202] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.276227] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.278330] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.280311] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.283147] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.285645] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.288336] NET: Registered protocol family 2 [ 1.291323] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.297844] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.303063] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.307924] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.312707] TCP: Hash tables configured (established 65536 bind 65536) [ 1.316397] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.320437] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.327647] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.334809] NET: Registered protocol family 1 [ 1.341599] RPC: Registered named UNIX socket transport module. [ 1.347820] RPC: Registered udp transport module. [ 1.352163] RPC: Registered tcp transport module. [ 1.355249] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.361580] NET: Registered protocol family 44 [ 1.365293] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.370841] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.373733] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.376301] PCI: CLS 0 bytes, default 64 [ 1.378827] Unpacking initramfs... [ 4.567263] debug: unmapping init [mem 0xffff9dec7cc54000-0xffff9dec7ffbffff] [ 4.576293] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 4.579620] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 4.585309] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 5.801299] Initialise system trusted keyrings [ 5.807732] Key type blacklist registered [ 5.812128] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 5.839546] zbud: loaded [ 5.849125] *** VALIDATE nfs *** [ 5.850524] *** VALIDATE nfs4 *** [ 5.857118] pstore: using deflate compression [ 5.867947] Platform Keyring initialized [ 6.195683] NET: Registered protocol family 38 [ 6.198513] Key type asymmetric registered [ 6.200870] Asymmetric key parser 'x509' registered [ 6.203979] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 6.208719] io scheduler mq-deadline registered [ 6.211736] io scheduler kyber registered [ 6.214290] io scheduler bfq registered [ 6.217484] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 6.220882] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 6.225771] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.230936] ACPI: Power Button [PWRF] [ 6.238664] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 6.249753] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 6.287354] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 6.315545] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 6.345468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.380603] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 6.413445] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.418640] Non-volatile memory driver v1.3 [ 6.420474] Linux agpgart interface v0.103 [ 6.460432] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 6.463487] vda: detected capacity change from 0 to 74698752 [ 6.483477] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 6.487506] vdb: detected capacity change from 0 to 1073741824 [ 6.520503] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.524309] vdc: detected capacity change from 0 to 2621440000 [ 6.549951] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 6.556360] vdd: detected capacity change from 0 to 2621440000 [ 6.603822] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.609835] vde: detected capacity change from 0 to 4294967296 [ 6.666247] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 6.676520] vdf: detected capacity change from 0 to 4294967296 [ 6.695745] libphy: Fixed MDIO Bus: probed [ 6.712346] usbcore: registered new interface driver usbserial_generic [ 6.716554] usbserial: USB Serial support registered for generic [ 6.718862] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 6.723776] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 6.728743] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 6.734937] mousedev: PS/2 mouse device common for all mice [ 6.742860] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 6.744840] rtc_cmos 00:05: RTC can wake from S4 [ 6.758147] rtc_cmos 00:05: registered as rtc0 [ 6.758323] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 6.760216] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 6.760287] intel_pstate: CPU model not supported [ 6.769671] hid: raw HID events driver (C) Jiri Kosina [ 6.782620] usbcore: registered new interface driver usbhid [ 6.790062] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 6.792850] usbhid: USB HID core driver [ 6.793290] drop_monitor: Initializing network drop monitor service [ 6.793558] Initializing XFRM netlink socket [ 6.794939] NET: Registered protocol family 10 [ 6.817295] Segment Routing with IPv6 [ 6.819711] NET: Registered protocol family 17 [ 6.822726] mpls_gso: MPLS GSO support [ 6.830763] RAS: Correctable Errors collector initialized. [ 6.836731] AVX version of gcm_enc/dec engaged. [ 6.841689] AES CTR mode by8 optimization enabled [ 7.060380] sched_clock: Marking stable (7060312226, 0)->(8689989477, -1629677251) [ 7.066724] registered taskstats version 1 [ 7.069731] Loading compiled-in X.509 certificates [ 7.073264] zswap: loaded using pool lzo/zbud [ 7.121739] Key type big_key registered [ 7.148741] Key type encrypted registered [ 7.152165] ima: No TPM chip found, activating TPM-bypass! [ 7.155612] ima: Allocated hash algorithm: sha1 [ 7.158324] ima: No architecture policies found [ 7.161220] evm: Initialising EVM extended attributes: [ 7.163234] evm: security.selinux [ 7.164729] evm: security.ima [ 7.166706] evm: security.capability [ 7.169135] evm: HMAC attrs: 0x1 [ 7.172996] rtc_cmos 00:05: setting system clock to 2026-08-08 10:40:11 UTC (1786185611) [ 7.184686] debug: unmapping init [mem 0xffffffffa4c03000-0xffffffffa4dfffff] [ 7.190983] debug: unmapping init [mem 0xffffffffa3982000-0xffffffffa3c58fff] [ 7.203504] Write protecting the kernel read-only data: 28672k [ 7.210966] debug: unmapping init [mem 0xffffffffa2003000-0xffffffffa21fffff] [ 7.216997] debug: unmapping init [mem 0xffffffffa2914000-0xffffffffa29fffff] [ 7.280566] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 7.302681] systemd[1]: Detected virtualization kvm. [ 7.306681] systemd[1]: Detected architecture x86-64. [ 7.309323] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 7.351807] systemd[1]: No hostname configured. [ 7.354054] systemd[1]: Set hostname to . [ 7.356960] random: systemd: uninitialized urandom read (16 bytes read) [ 7.362175] systemd[1]: Initializing machine ID from random generator. [ 7.516309] random: ln: uninitialized urandom read (6 bytes read) [ 7.780635] random: systemd: uninitialized urandom read (16 bytes read) [ 7.788499] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 7.821599] systemd[1]: Started Memstrack Anylazing Service. [ OK ] Started Memstrack Anylazing Service. [ 7.884815] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Timers. Starting Journal Service... [ OK ] Reached target Swap. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Reached target Slices. [ OK ] Reached target Paths. Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 9.360725] device-mapper: uevent: version 1.0.3 [ 9.365923] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 11.824800] random: fast init done [ 11.989733] scsi host0: ata_piix [ 11.993618] virtio_net virtio0 ens2: renamed from eth0 [ 12.069518] scsi host1: ata_piix [ 12.092704] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 12.106416] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 17.679676] random: crng init done [ 17.681112] random: 7 urandom warning(s) missed due to ratelimiting [ 20.271270] dracut-initqueue[582]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 22.472256] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Timers. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Swap. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 26.327503] printk: systemd: 26 output lines suppressed due to ratelimiting [ 27.177347] SELinux: Disabled at runtime. [ 27.319775] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 27.342684] systemd[1]: Detected virtualization kvm. [ 27.346611] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 28.914745] systemd[1]: initrd-switch-root.service: Succeeded. [ 28.920901] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 28.935329] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 28.957320] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 28.966491] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 28.982765] systemd[1]: Starting Journal Service... Starting Journal Service... [ 28.992893] systemd[1]: Mounting Huge Pages File System... Mounting Huge Pages File System... [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Listening on udev Control Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Slices. [ OK ] Reached target Paths. Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 29.501434] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-getty.slice. Mounting POSIX Message Queue File System... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on udev Kernel Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Mounted POSIX Message Queue File System. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 31.277130] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 32.772414] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 33.196703] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 34.000985] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 34.101470] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 38.993227] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit)[ 39.434586] NFS: Registering the id_resolver key type [ 39.436639] Key type id_resolver registered [ 39.438695] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (11s / no limit) [ **] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... Starting RPC Bind... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Login Service... [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg406-server login: [ 70.422411] hrtimer: interrupt took 2676278 ns [ 108.128746] libcfs: loading out-of-tree module taints kernel. [ 108.192542] Key type ._llcrypt registered [ 108.195613] Key type .llcrypt registered [ 108.279199] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 128.237593] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 130.844560] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 130.863631] alg: No test for adler32 (adler32-zlib) [ 132.374735] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 133.445862] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 135.207178] Key type lgssc registered [ 137.915709] Lustre: Echo OBD driver; http://www.lustre.org/ [ 155.602254] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 207.382285] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 223.593103] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 223.640092] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 224.881262] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 224.904748] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 224.959746] Lustre: lustre-MDT0000: new disk, initializing [ 225.036745] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 225.054797] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 230.964816] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 247.204162] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 247.373048] Lustre: 6533:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 247.421459] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 247.428256] Lustre: Skipped 1 previous similar message [ 247.505215] Lustre: lustre-MDT0001: new disk, initializing [ 247.582533] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 247.608388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 247.632848] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 254.926597] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 260.607371] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 273.107277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 273.348571] Lustre: lustre-OST0000: new disk, initializing [ 273.352126] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 273.357395] Lustre: 8472:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 273.428044] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 281.062746] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 283.200531] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 283.213451] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 283.370814] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 299.254051] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 299.476564] Lustre: lustre-OST0001: new disk, initializing [ 299.479877] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 299.490760] Lustre: 9544:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 299.591989] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 305.221102] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 305.243772] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 305.452599] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 307.322889] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 320.883405] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 333.166625] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 343.530714] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing check_logdir /tmp/testlogs/ [ 350.636296] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing yml_node [ 357.586307] Lustre: DEBUG MARKER: Client: 2.17.56.50 [ 361.048404] Lustre: DEBUG MARKER: MDS: 2.17.56.50 [ 363.780729] Lustre: DEBUG MARKER: OSS: 2.17.56.50 [ 366.021348] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Sat Aug 8 06:46:08 EDT 2026 [ 388.888515] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 391.369524] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 411.621626] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 411.628194] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 411.644456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 413.153924] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 413.154604] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 414.753121] Lustre: server umount lustre-MDT0000 complete [ 419.072724] LustreError: 6524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186023 with bad export cookie 5047538171768375389 [ 419.094179] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 419.098931] LustreError: 6524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 423.392331] LustreError: 9551:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 423.393247] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 423.395120] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 423.395128] Lustre: Skipped 3 previous similar messages [ 423.410536] LustreError: 9551:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 423.443571] Lustre: Skipped 3 previous similar messages [ 424.421686] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 424.436511] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 425.568615] Lustre: server umount lustre-MDT0001 complete [ 431.236705] Lustre: server umount lustre-OST0000 complete [ 436.293574] Lustre: server umount lustre-OST0001 complete [ 444.070683] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 459.585545] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 508.850430] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 509.358441] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 509.404684] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 509.493866] Lustre: lustre-MDT0000: new disk, initializing [ 509.636716] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 509.685630] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 514.353287] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 526.497170] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 526.696882] Lustre: 15123:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 526.787731] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 526.793131] Lustre: Skipped 1 previous similar message [ 526.891778] Lustre: lustre-MDT0001: new disk, initializing [ 527.050096] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 527.110090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 527.132582] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 532.241596] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 544.645440] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 552.173616] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 563.039971] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 563.429793] Lustre: lustre-OST0000: new disk, initializing [ 563.433394] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 563.440414] Lustre: 17097:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 563.558138] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 564.897114] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 564.919094] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 565.091102] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 571.481807] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 584.599610] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 591.253689] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 591.511175] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 597.650674] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 597.877633] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 601.058786] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 601.059876] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 601.077483] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 601.092687] Lustre: Skipped 1 previous similar message [ 601.107602] Lustre: Skipped 3 previous similar messages [ 606.181559] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 606.198605] Lustre: Skipped 1 previous similar message [ 606.441669] Lustre: server umount lustre-OST0000 complete [ 611.296979] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 616.426118] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 616.453363] Lustre: Skipped 3 previous similar messages [ 625.119391] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 625.385565] Lustre: server umount lustre-MDT0000 complete [ 626.659234] LustreError: 15136:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 626.701683] LustreError: 15136:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 630.219976] LustreError: 15117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186234 with bad export cookie 5047538171768376929 [ 630.228859] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 630.229404] LustreError: 15117:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 630.523545] Lustre: server umount lustre-MDT0001 complete [ 637.874660] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:50:39 (1786186239) [ 645.499721] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 646.028144] LustreError: 19367:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 646.142362] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 651.267151] LustreError: 19368:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 652.936479] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 662.511397] LustreError: 19367:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 662.547836] LustreError: 19367:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 664.464402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 672.479357] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 684.537963] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 694.079201] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 703.415607] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 713.244629] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 722.809794] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 727.150466] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 736.736374] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 736.749488] LustreError: Skipped 1 previous similar message [ 736.758339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 736.773991] Lustre: Skipped 1 previous similar message [ 736.783183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 736.792470] Lustre: Skipped 2 previous similar messages [ 740.818103] Lustre: server umount lustre-MDT0000 complete [ 742.887207] LustreError: 19368:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 742.908165] LustreError: 19368:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 746.176070] LustreError: 19348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186350 with bad export cookie 5047538171768377811 [ 746.191858] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 746.194496] LustreError: 19348:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 746.567952] Lustre: server umount lustre-MDT0001 complete [ 761.095440] Lustre: server umount lustre-OST0000 complete [ 792.740312] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 807.179806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 807.777956] LustreError: 23757:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 807.920909] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 807.937123] Lustre: Skipped 2 previous similar messages [ 815.784297] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 827.012736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 834.684394] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 838.823963] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 848.873440] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 858.643394] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 870.617236] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 870.950341] Lustre: lustre-OST0001: new disk, initializing [ 870.953690] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 870.962337] Lustre: 26075:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 872.351644] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 872.397154] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 872.667723] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 883.212212] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 894.131899] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 899.539406] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 919.019400] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 919.035050] Lustre: Skipped 4 previous similar messages [ 919.040204] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 919.044411] Lustre: Skipped 4 previous similar messages [ 921.366416] Lustre: server umount lustre-MDT0000 complete [ 924.133979] LustreError: 23758:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 924.147212] LustreError: 23758:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 926.267354] LustreError: 23737:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186530 with bad export cookie 5047538171768378658 [ 926.278051] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 926.660432] Lustre: server umount lustre-MDT0001 complete [ 942.303339] Lustre: server umount lustre-OST0000 complete [ 945.635376] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786186533/real 1786186533] req@ffff9debc901bb80 x1872951495635840/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786186549 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 947.160299] Lustre: server umount lustre-OST0001 complete [ 970.324320] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 984.264458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 985.203434] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 985.227438] Lustre: Skipped 3 previous similar messages [ 990.613133] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1000.782118] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1006.735610] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1011.193682] Lustre: 30249:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1021.197257] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1029.830724] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1040.800893] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1049.259852] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1060.314307] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1065.045920] Lustre: 32129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1073.131822] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1073.156284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1073.199447] Lustre: Skipped 2 previous similar messages [ 1073.224192] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1079.646653] Lustre: server umount lustre-MDT0000 complete [ 1082.349761] LustreError: 29096:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1082.391919] LustreError: 29096:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 1085.301139] LustreError: 29080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186689 with bad export cookie 5047538171768379953 [ 1085.315604] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1085.338894] LustreError: 29080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1085.979668] Lustre: server umount lustre-MDT0001 complete [ 1101.015549] Lustre: server umount lustre-OST0000 complete [ 1103.585851] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786186691/real 1786186691] req@ffff9decd0846300 x1872951495693440/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786186707 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1105.888332] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786186694/real 1786186694] req@ffff9decd0879c00 x1872951495693696/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786186710 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1106.496516] Lustre: server umount lustre-OST0001 complete [ 1119.696676] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 06:58:41 (1786186721) [ 1154.226219] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1168.874336] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1175.169733] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1188.901881] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1200.497865] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1207.365843] Lustre: 36088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1217.380154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1227.869901] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1240.846875] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1241.246251] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1241.254378] Lustre: Skipped 6 previous similar messages [ 1251.071790] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1262.298833] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1267.322510] Lustre: 37979:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1276.900777] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1276.929888] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1276.952214] Lustre: Skipped 4 previous similar messages [ 1276.970729] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1276.981975] Lustre: Skipped 4 previous similar messages [ 1280.396275] Lustre: server umount lustre-MDT0000 complete [ 1285.581138] LustreError: 34917:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786186889 with bad export cookie 5047538171768381059 [ 1285.605166] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1285.981593] Lustre: server umount lustre-MDT0001 complete [ 1302.151887] Lustre: server umount lustre-OST0000 complete [ 1302.751181] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786186891/real 1786186891] req@ffff9debc7c81f80 x1872951495753216/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786186907 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1307.105554] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786186895/real 1786186895] req@ffff9deceeb19880 x1872951495753472/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786186911 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1309.044269] Lustre: server umount lustre-OST0001 complete [ 1322.381347] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 07:02:04 (1786186924) [ 1346.782177] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1362.330645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.840563] LustreError: 40146:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1362.859297] LustreError: 40146:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 1368.896096] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1381.670516] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1389.207922] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1393.293513] Lustre: 41297:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1401.546897] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1410.243882] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1421.061845] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1429.932755] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1439.414290] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1443.836342] Lustre: 43176:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1445.101373] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1445.110546] Lustre: Skipped 1 previous similar message [ 1448.928946] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1448.937263] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1448.947666] Lustre: Skipped 4 previous similar messages [ 1455.268439] Lustre: server umount lustre-MDT0000 complete [ 1460.099713] LustreError: 40128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187064 with bad export cookie 5047538171768382165 [ 1460.107660] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1460.116461] LustreError: 40128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 1466.546193] Lustre: server umount lustre-MDT0001 complete [ 1477.090241] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1485.283510] LustreError: 3641:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff9decfeb36300 x1872951495811328/t0(0) o250->MGC192.168.204.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1492.842556] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1495.496922] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1502.825783] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1508.350773] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1508.355509] Lustre: Skipped 1 previous similar message [ 1509.403162] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1519.833171] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1522.598643] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1530.966562] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1532.893554] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1539.058753] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1539.066172] Lustre: Skipped 16 previous similar messages [ 1539.067423] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1545.003142] Lustre: server umount lustre-MDT0000 complete [ 1549.739780] LustreError: 40126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187154 with bad export cookie 5047538171768383264 [ 1549.743921] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1549.750484] LustreError: 40126:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1550.131829] Lustre: server umount lustre-MDT0001 complete [ 1565.496569] Lustre: server umount lustre-OST0000 complete [ 1578.941514] Lustre: server umount lustre-OST0001 complete [ 1591.720328] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 07:06:33 (1786187193) [ 1612.089241] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1623.197696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1630.731109] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1642.469917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1650.281588] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1654.405392] Lustre: 48939:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1663.699277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1673.588907] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1684.940395] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1694.178561] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1705.113653] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1711.869446] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1731.558723] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1731.592017] Lustre: Skipped 10 previous similar messages [ 1736.593579] Lustre: server umount lustre-MDT0000 complete [ 1742.549881] LustreError: 47770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786187346 with bad export cookie 5047538171768384083 [ 1742.563775] LustreError: 47770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1742.568639] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1743.056345] Lustre: server umount lustre-MDT0001 complete [ 1758.525211] Lustre: server umount lustre-OST0000 complete [ 1763.298210] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786187351/real 1786187351] req@ffff9debc1635f80 x1872951495922688/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786187367 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1764.305899] Lustre: server umount lustre-OST0001 complete [ 1778.987926] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:09:40 (1786187380) [ 1782.417889] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1784.863575] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:09:47 (1786187387) [ 1798.238934] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 1810.981822] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 1842.421230] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1842.797866] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1842.807352] Lustre: Skipped 1 previous similar message [ 1842.955935] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1843.133375] Lustre: lustre-MDT0000: new disk, initializing [ 1843.258481] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1843.273671] Lustre: Skipped 10 previous similar messages [ 1843.317804] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1850.526732] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1862.440273] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1871.305732] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1871.627968] Lustre: lustre-OST0000: new disk, initializing [ 1871.635444] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1871.642699] Lustre: Skipped 1 previous similar message [ 1871.652977] Lustre: 55050:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1873.307792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1873.320476] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1873.382707] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1880.177409] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 1893.443476] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1896.288957] Lustre: DEBUG MARKER: create test files [ 1929.696872] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1933.930898] Lustre: server umount lustre-OST0000 complete [ 1938.889898] Lustre: server umount lustre-MDT0000 complete [ 1952.721616] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 1956.252413] Key type lgssc unregistered [ 1956.799474] LNet: 56687:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1956.815144] LNetError: 56687:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1956.846557] LNet: Removed LNI 192.168.204.106@tcp [ 1958.314170] Key type .llcrypt unregistered [ 1958.319293] Key type ._llcrypt unregistered [ 1997.809398] Key type ._llcrypt registered [ 1997.813832] Key type .llcrypt registered [ 1997.926440] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2000.224479] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2000.365027] alg: No test for adler32 (adler32-zlib) [ 2002.309340] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 2002.971826] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2004.695186] Key type lgssc registered [ 2006.537039] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2017.738394] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2017.766243] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2019.149062] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2025.451715] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2035.888580] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2044.541368] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2044.951508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2051.556567] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2062.308936] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2064.378956] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2064.482328] Lustre: 59436:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 2064.489129] Lustre: 59436:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 2064.493992] Lustre: 59436:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2064.500101] Lustre: 59436:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2064.505377] Lustre: 59436:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2064.510397] Lustre: 59436:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2075.850171] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2084.295680] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2095.079426] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2095.091088] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2095.113680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2099.726579] Lustre: server umount lustre-OST0000 complete [ 2105.618797] Lustre: server umount lustre-MDT0000 complete [ 2124.434130] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2128.937413] Key type lgssc unregistered [ 2129.419731] LNet: 61094:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2129.434679] LNetError: 61094:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2129.454723] LNet: Removed LNI 192.168.204.106@tcp [ 2130.738177] Key type .llcrypt unregistered [ 2130.739951] Key type ._llcrypt unregistered [ 2171.764879] Key type ._llcrypt registered [ 2171.769836] Key type .llcrypt registered [ 2171.922111] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2174.655410] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2174.881128] alg: No test for adler32 (adler32-zlib) [ 2176.109535] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 2176.356285] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2178.055238] Key type lgssc registered [ 2180.109428] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2190.891989] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2190.935240] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2192.572706] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2198.042908] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2207.748198] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2218.037886] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2218.381460] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2226.067336] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2235.027460] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2236.742763] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2236.888916] Lustre: 63927:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 2236.894077] Lustre: 63927:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 2236.898736] Lustre: 63927:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2236.902938] Lustre: 63927:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2236.913709] Lustre: 63927:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2236.920130] Lustre: 63927:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2248.528682] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2262.499956] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2262.507097] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2262.538463] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2265.762366] Lustre: server umount lustre-OST0000 complete [ 2270.810493] Lustre: server umount lustre-MDT0000 complete [ 2288.713827] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 2291.986416] Key type lgssc unregistered [ 2292.424667] LNet: 65533:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2292.438470] LNetError: 65533:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2293.478803] LNet: Removed LNI 192.168.204.106@tcp [ 2294.446080] Key type .llcrypt unregistered [ 2294.449081] Key type ._llcrypt unregistered [ 2319.574921] Key type ._llcrypt registered [ 2319.582516] Key type .llcrypt registered [ 2319.838619] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 2345.747182] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2348.201569] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2348.238352] alg: No test for adler32 (adler32-zlib) [ 2349.470680] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 2349.690765] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 2351.383164] Key type lgssc registered [ 2353.415834] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2400.050692] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2400.106335] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2401.569222] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2401.613810] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2401.727606] Lustre: lustre-MDT0000: new disk, initializing [ 2401.916821] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2401.950705] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2408.126455] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2422.090552] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2422.225840] Lustre: 68693:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2422.267625] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2422.271577] Lustre: Skipped 1 previous similar message [ 2422.339508] Lustre: lustre-MDT0001: new disk, initializing [ 2422.461587] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2422.537635] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2422.582671] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2429.031459] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2441.693865] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2451.141821] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2459.953828] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2460.262929] Lustre: lustre-OST0000: new disk, initializing [ 2460.274909] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2460.315250] Lustre: 70665:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2460.548788] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2466.371946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2466.382345] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2466.658489] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2468.714312] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2480.246573] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2485.666752] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2485.918407] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2490.957565] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2491.186839] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2497.505543] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2497.512100] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2497.520203] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2497.529921] Lustre: Skipped 1 previous similar message [ 2499.672573] Lustre: server umount lustre-OST0000 complete [ 2504.673636] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2504.681588] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2504.707260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2504.712910] Lustre: Skipped 1 previous similar message [ 2507.746938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2507.749702] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2510.492807] Lustre: server umount lustre-MDT0000 complete [ 2516.182619] LustreError: 69487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188120 with bad export cookie 3798871108700998449 [ 2516.185529] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2516.201207] LustreError: 69487:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2516.572910] Lustre: server umount lustre-MDT0001 complete [ 2527.825462] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:22:10 (1786188130) [ 2548.823937] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2608.586343] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 2626.490723] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 2679.580757] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2680.059454] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2680.099452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2680.202317] Lustre: lustre-MDT0000: new disk, initializing [ 2680.338417] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2680.377178] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2686.393303] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2700.883593] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2701.048440] Lustre: 76734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2701.137350] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2701.142671] Lustre: Skipped 1 previous similar message [ 2701.235721] Lustre: lustre-MDT0001: new disk, initializing [ 2701.322693] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2701.368527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2701.387736] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2707.840312] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2722.346239] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2730.205545] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2737.780896] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2738.030377] Lustre: lustre-OST0000: new disk, initializing [ 2738.033880] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2738.040767] Lustre: 78704:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2738.143976] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2744.042546] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2744.107754] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2744.461281] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2745.403722] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2758.767312] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2765.269115] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2765.643238] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2772.018625] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2772.373582] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2780.127811] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2780.135581] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2780.137699] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2780.137706] Lustre: Skipped 1 previous similar message [ 2780.138167] LustreError: Skipped 1 previous similar message [ 2780.189811] Lustre: Skipped 1 previous similar message [ 2781.448597] Lustre: server umount lustre-OST0000 complete [ 2788.326720] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2788.338460] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2788.368277] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2788.380577] Lustre: Skipped 1 previous similar message [ 2792.572286] Lustre: server umount lustre-MDT0000 complete [ 2795.495696] LustreError: 76745:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2795.555522] LustreError: 76745:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2797.937193] LustreError: 76725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188402 with bad export cookie 3798871108700999338 [ 2797.944084] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2797.954167] LustreError: 76725:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2798.276843] Lustre: server umount lustre-MDT0001 complete [ 2810.778321] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:26:52 (1786188412) [ 2819.843912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2820.240835] LustreError: 81118:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2820.360480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2825.701315] LustreError: 81119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2827.211064] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2829.800311] LustreError: 81118:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2834.917492] LustreError: 81119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2838.209400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2844.058415] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2855.058538] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2863.616680] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2874.559650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2875.143990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2875.151621] Lustre: Skipped 1 previous similar message [ 2883.730452] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2894.090966] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2898.632114] Lustre: 83796:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 26f7cf59-8f03-43c6-8cac-ca0e2e62fe08 at adminstrative request [ 2902.001789] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2902.030992] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2912.230747] Lustre: MGS: Client 26f7cf59-8f03-43c6-8cac-ca0e2e62fe08 (at 0@lo) reconnecting [ 2912.249857] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x34b84d25659e121c to 0x34b84d25659e14bc [ 2912.283551] Lustre: MGC192.168.204.106@tcp: Connection restored to 0@lo (at 0@lo) [ 2938.537558] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2938.690435] Lustre: lustre-OST0001: new disk, initializing [ 2938.695622] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2938.701535] Lustre: 84187:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2947.268298] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 2948.880609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2948.918976] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2949.063736] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2959.487785] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2964.450090] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2964.455157] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2964.463233] LustreError: Skipped 1 previous similar message [ 2964.466425] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2964.471387] Lustre: Skipped 2 previous similar messages [ 2964.497885] Lustre: Skipped 3 previous similar messages [ 2977.247192] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2977.361874] Lustre: server umount lustre-OST0001 complete [ 2979.833377] LustreError: 82942:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2979.879701] LustreError: 82942:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2983.907506] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2983.915520] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2983.928336] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2983.935477] Lustre: Skipped 4 previous similar messages [ 2997.221777] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2997.481230] Lustre: server umount lustre-OST0000 complete [ 3002.336641] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3007.994288] Lustre: server umount lustre-MDT0000 complete [ 3010.542907] LustreError: 81119:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3010.599595] LustreError: 81119:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3012.424875] LustreError: 81100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188616 with bad export cookie 3798871108701000892 [ 3012.426375] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3012.443204] LustreError: 81100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3012.698354] Lustre: server umount lustre-MDT0001 complete [ 3025.816585] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 07:30:27 (1786188627) [ 3029.020280] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3032.299958] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 07:30:33 (1786188633) [ 3035.473053] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3039.309844] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 07:30:40 (1786188640) [ 3041.521885] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3044.683879] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 07:30:46 (1786188646) [ 3047.469799] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3050.558823] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 07:30:52 (1786188652) [ 3053.518408] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3062.134836] loop: module loaded [ 3062.152878] loop0: detected capacity change from 0 to 4294967296 [ 3062.246621] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3064.558590] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3068.145652] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3072.212792] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3110.605383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3110.676199] LustreError: 88319:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3110.695766] LustreError: 88319:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3110.704552] LustreError: 88319:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3110.718575] LustreError: 88319:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3139.015976] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3155.874299] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3197.829542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3198.196708] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3198.246570] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3198.345142] Lustre: lustre-MDT0000: new disk, initializing [ 3198.442296] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3198.449840] Lustre: Skipped 1 previous similar message [ 3198.466305] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3204.604876] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3216.444818] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3216.542519] Lustre: 91210:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3216.905224] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3222.175474] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3233.329288] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3240.205842] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3249.105248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3249.450177] Lustre: 93182:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3256.579509] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3256.900204] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3258.846568] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3272.824868] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3279.279873] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3279.840833] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3284.554533] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3284.841636] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3287.520403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3287.528388] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3287.539069] LustreError: Skipped 1 previous similar message [ 3287.540926] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3287.560283] Lustre: Skipped 4 previous similar messages [ 3287.567131] Lustre: Skipped 9 previous similar messages [ 3301.343489] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3301.609858] Lustre: server umount lustre-OST0000 complete [ 3312.554516] Lustre: server umount lustre-MDT0000 complete [ 3313.123775] LustreError: 91220:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3313.149562] LustreError: 91220:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3317.784260] LustreError: 91200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786188922 with bad export cookie 3798871108701001508 [ 3317.804396] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3317.806863] LustreError: 91200:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3318.215736] Lustre: server umount lustre-MDT0001 complete [ 3330.055880] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 07:35:31 (1786188931) [ 3338.037283] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3338.530479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3338.540870] Lustre: Skipped 2 previous similar messages [ 3344.599744] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3354.419738] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3360.711622] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3373.058160] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3381.561963] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3390.671244] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3398.393946] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3408.450558] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3414.417405] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3414.792498] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3423.515186] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3423.902811] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3432.381633] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3432.691732] Lustre: lustre-OST0001: new disk, initializing [ 3432.694741] Lustre: Skipped 2 previous similar messages [ 3432.699056] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3432.706839] Lustre: Skipped 3 previous similar messages [ 3432.731939] Lustre: 98970:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3440.244426] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3440.816947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3440.832414] Lustre: Skipped 2 previous similar messages [ 3440.837879] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3440.960432] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3452.174717] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3457.840684] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3458.217600] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3463.103744] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3463.343500] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3489.396910] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3521.679675] Lustre: Failing over lustre-MDT0000 [ 3522.135555] Lustre: server umount lustre-MDT0000 complete [ 3522.168215] LustreError: 98077:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3522.180537] LustreError: 98077:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3523.045107] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3523.064539] Lustre: Skipped 4 previous similar messages [ 3539.487178] Lustre: 66914:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786189127/real 1786189127] req@00000000bf8a7a69 x1872953820365952/t0(0) o400->MGC192.168.204.106@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786189143 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3539.544861] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3549.319742] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3549.328165] LDISKFS-fs (dm-0): recovery complete [ 3549.351842] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3564.529562] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x34b84d25659edf37 [ 3564.561785] Lustre: MGC192.168.204.106@tcp: Connection restored to 0@lo (at 0@lo) [ 3564.927750] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3568.234262] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3568.237066] Lustre: 97438:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3570.157139] Lustre: 96324:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3570.182283] Lustre: 96324:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 3570.225460] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3570.305805] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3570.631124] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3570.966647] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3570.984624] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3571.315092] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3571.333610] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 3571.344917] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3571.364550] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 3571.587849] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3572.041431] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3572.053690] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3572.392515] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3572.406083] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 3573.412470] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3573.439579] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 3574.108457] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3574.131520] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 3574.477087] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3574.510878] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 3577.715909] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3577.734162] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 3578.387309] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3578.403563] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3578.719223] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3578.728440] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 3586.026907] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3586.053382] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 3586.697852] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3586.725420] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 3587.071138] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3587.084713] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 3602.296067] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3602.311274] Lustre: 101828:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 3602.957979] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3602.982206] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3603.311322] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3603.319403] LustreError: 101828:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3628.287162] Lustre: 101828:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3628.297055] Lustre: 101828:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3628.303131] Lustre: 101828:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 1b670409-d4da-4b09-805e-fd0687a36aa0@192.168.204.6@tcp [ 3628.312766] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3628.317805] Lustre: 101828:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000220121ce x1872953791814656/t0(8589934778) o35->1b670409-d4da-4b09-805e-fd0687a36aa0@192.168.204.6@tcp:428/0 lens 392/0 e 0 to 0 dl 1786189243 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3628.352709] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3628.376589] Lustre: lustre-MDT0000: Denying connection for new client 1b670409-d4da-4b09-805e-fd0687a36aa0 (at 192.168.204.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3628.427071] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3628.495136] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3628.504075] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3628.509248] Lustre: Skipped 2 previous similar messages [ 3628.559184] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3628.559589] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3646.954314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3646.961650] Lustre: Skipped 8 previous similar messages [ 3651.751760] Lustre: server umount lustre-OST0000 complete [ 3652.072835] LustreError: 97417:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3652.079836] LustreError: 97417:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 59 previous similar messages [ 3657.206271] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3657.232639] Lustre: Skipped 2 previous similar messages [ 3662.434537] Lustre: server umount lustre-OST0001 complete [ 3669.477324] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3683.295134] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3683.744615] Lustre: server umount lustre-MDT0000 complete [ 3689.748399] LustreError: 98968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189294 with bad export cookie 3798871108701052727 [ 3689.757241] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3689.761350] LustreError: 98968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3690.161804] Lustre: server umount lustre-MDT0001 complete [ 3702.965533] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:41:45 (1786189305) [ 3713.588858] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3713.597842] Lustre: *** cfs_fail_loc=197, val=0*** [ 3729.183749] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3734.303468] LustreError: 103805:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 3739.540614] Lustre: *** cfs_fail_loc=197, val=0*** [ 3739.544804] Lustre: Skipped 65 previous similar messages [ 3760.159431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3760.176699] Lustre: Skipped 4 previous similar messages [ 3769.249579] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3779.913995] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3782.590089] Lustre: server umount lustre-OST0000 complete [ 3795.912977] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:43:17 (1786189397) [ 3821.365072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3827.542630] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3838.794226] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3843.313542] LustreError: 105458:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3843.328443] LustreError: 105458:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3843.341986] LustreError: 105458:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3843.668377] Lustre: server umount lustre-MDT0000 complete [ 3889.073974] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 3907.096431] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 3960.007870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3960.335486] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3960.372983] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3960.454857] Lustre: lustre-MDT0000: new disk, initializing [ 3960.547486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3968.032085] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 3983.044554] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3983.366232] Lustre: 109659:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3984.081048] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3990.859624] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4003.328500] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4011.684601] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4021.796751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4022.069749] Lustre: 111627:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4029.228473] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4029.526247] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4042.722950] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4049.550430] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4049.786889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4056.230256] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4056.530942] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4060.128096] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4060.137137] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4060.156255] LustreError: Skipped 1 previous similar message [ 4060.161873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4060.161882] Lustre: Skipped 9 previous similar messages [ 4060.234453] Lustre: Skipped 3 previous similar messages [ 4065.800095] Lustre: server umount lustre-OST0000 complete [ 4076.971253] Lustre: server umount lustre-MDT0000 complete [ 4080.610215] LustreError: 109665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4080.641293] LustreError: 109665:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 4081.351365] LustreError: 109651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189685 with bad export cookie 3798871108701063402 [ 4081.370644] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4081.374735] LustreError: 109651:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4081.844917] Lustre: server umount lustre-MDT0001 complete [ 4095.154954] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:48:16 (1786189696) [ 4109.721950] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4113.494579] Key type lgssc unregistered [ 4114.038825] LNet: 114118:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4114.050995] LNetError: 114118:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4114.065968] LNet: Removed LNI 192.168.204.106@tcp [ 4115.176882] Key type .llcrypt unregistered [ 4115.180233] Key type ._llcrypt unregistered [ 4143.389613] Key type ._llcrypt registered [ 4143.392274] Key type .llcrypt registered [ 4143.564539] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4168.038169] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4169.748152] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4169.840748] alg: No test for adler32 (adler32-zlib) [ 4171.565775] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 4171.915352] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4173.712770] Key type lgssc registered [ 4175.367829] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4231.901300] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4231.930048] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4233.490535] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4233.546702] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4233.664676] Lustre: lustre-MDT0000: new disk, initializing [ 4233.780940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4233.834526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4242.225912] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4257.811521] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4258.074086] Lustre: lustre-OST0000: new disk, initializing [ 4258.077892] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4258.082511] Lustre: Skipped 1 previous similar message [ 4258.086690] Lustre: 118300:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4258.167420] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4267.799779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4267.834427] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4267.920315] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4269.267441] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4285.925325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4286.321153] Lustre: lustre-OST0001: new disk, initializing [ 4286.324704] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4286.338931] Lustre: 119164:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4286.565344] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4294.512581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4294.558617] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4294.615327] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4296.418229] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4309.986235] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4309.988988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4310.006601] Lustre: Skipped 1 previous similar message [ 4310.045641] Lustre: Skipped 1 previous similar message [ 4313.829846] Lustre: server umount lustre-MDT0000 complete [ 4322.158872] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4329.037551] LustreError: 117485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786189933 with bad export cookie 5301421870746123023 [ 4329.050330] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4329.057592] LustreError: 117485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4329.258308] Lustre: server umount lustre-OST0000 complete [ 4334.315496] Lustre: server umount lustre-OST0001 complete [ 4348.589582] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 07:52:29 (1786189949) [ 4381.214949] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 07:53:02 (1786189982) [ 4401.191814] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4441.810498] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:54:03 (1786190043) [ 4502.370417] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 4517.068331] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4571.275742] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4571.693610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4571.798984] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4572.038995] Lustre: lustre-MDT0000: new disk, initializing [ 4572.218304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4572.284840] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4578.870647] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4594.847258] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4595.129337] Lustre: 126959:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4595.230738] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4595.233649] Lustre: Skipped 1 previous similar message [ 4595.372901] Lustre: lustre-MDT0001: new disk, initializing [ 4595.548254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4595.639517] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4595.659111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4602.963863] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4615.888590] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4624.641648] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4635.281509] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4635.723544] Lustre: lustre-OST0000: new disk, initializing [ 4635.744577] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4635.765942] Lustre: 128928:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4635.939498] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4642.401845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4642.409697] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4642.593766] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4643.897753] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4656.909858] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4663.715377] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4664.230448] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4671.447421] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4671.924628] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4678.637956] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4678.638198] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4678.653361] LustreError: Skipped 1 previous similar message [ 4678.660141] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4683.761656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4683.779603] Lustre: Skipped 2 previous similar messages [ 4688.875324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4688.892392] Lustre: Skipped 1 previous similar message [ 4689.375133] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4689.601241] Lustre: server umount lustre-OST0000 complete [ 4698.081397] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4698.095045] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4698.115255] Lustre: Skipped 1 previous similar message [ 4698.124923] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4701.783996] Lustre: server umount lustre-MDT0000 complete [ 4706.718154] LustreError: 128927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786190311 with bad export cookie 5301421870746124248 [ 4706.731570] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4706.748034] LustreError: 128927:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4707.267465] Lustre: server umount lustre-MDT0001 complete [ 4721.512600] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:58:42 (1786190322) [ 4730.505615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4730.999756] LustreError: 131345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4731.033868] LustreError: 131345:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 4731.132350] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4736.480913] LustreError: 131346:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4736.490822] LustreError: 131346:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4737.787628] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4741.602327] LustreError: 131345:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4746.722466] LustreError: 131346:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4749.429623] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4756.904986] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4767.396445] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4776.578582] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4786.167582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4786.746669] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4786.763553] Lustre: Skipped 1 previous similar message [ 4795.648403] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4806.046755] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4812.194140] Lustre: *** cfs_fail_loc=159, val=0*** [ 4812.205486] Lustre: Skipped 1 previous similar message [ 4828.917158] Lustre: lustre-MDT0000: Client d7d1f4f4-7be1-483b-8812-c47310aa7c4c (at 192.168.204.6@tcp) reconnecting [ 4831.331770] Lustre: *** cfs_fail_loc=159, val=0*** [ 4831.346358] Lustre: Skipped 4 previous similar messages [ 4847.762443] Lustre: lustre-MDT0000: Client d7d1f4f4-7be1-483b-8812-c47310aa7c4c (at 192.168.204.6@tcp) reconnecting [ 4867.045634] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4867.056718] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4867.081790] Lustre: Skipped 1 previous similar message [ 4867.093227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4867.110251] Lustre: Skipped 2 previous similar messages [ 4871.234456] Lustre: server umount lustre-OST0000 complete [ 4877.797750] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4877.832196] LustreError: Skipped 1 previous similar message [ 4877.848591] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4877.873537] Lustre: Skipped 1 previous similar message [ 4883.450905] LustreError: 133887:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4883.492414] Lustre: server umount lustre-MDT0000 complete [ 4888.680022] LustreError: 131327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786190493 with bad export cookie 5301421870746125130 [ 4888.697645] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4888.702226] LustreError: 131327:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4889.130752] Lustre: server umount lustre-MDT0001 complete [ 4903.087789] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 4905.855657] Key type lgssc unregistered [ 4906.167921] LNet: 135144:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4906.181222] LNetError: 135144:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4906.196225] LNet: Removed LNI 192.168.204.106@tcp [ 4907.404937] Key type .llcrypt unregistered [ 4907.407538] Key type ._llcrypt unregistered [ 4945.371861] Key type ._llcrypt registered [ 4945.374742] Key type .llcrypt registered [ 4945.517878] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 4947.362656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4947.422982] alg: No test for adler32 (adler32-zlib) [ 4948.967456] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 4949.296851] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 4951.015399] Key type lgssc registered [ 4952.776944] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4964.513547] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4964.526469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4965.887288] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4973.445519] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 4976.106851] LustreError: 136564:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4976.141913] LustreError: 136564:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4981.239058] LustreError: 136565:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4986.355955] LustreError: 136564:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4986.522940] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4987.112175] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4995.170689] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5005.185639] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5012.994929] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5023.257660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5024.119043] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5032.868415] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5036.495107] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5044.491339] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5056.010373] Lustre: *** cfs_fail_loc=159, val=0*** [ 5072.532323] Lustre: lustre-MDT0000: Client 1688bc56-93f7-4322-8829-3d50d7496c5f (at 192.168.204.6@tcp) reconnecting [ 5076.504183] Lustre: *** cfs_fail_loc=159, val=0*** [ 5093.062979] Lustre: lustre-MDT0001: Client 1688bc56-93f7-4322-8829-3d50d7496c5f (at 192.168.204.6@tcp) reconnecting [ 5095.412602] Lustre: *** cfs_fail_loc=159, val=0*** [ 5095.422108] Lustre: Skipped 3 previous similar messages [ 5111.960673] Lustre: lustre-MDT0001: Client 1688bc56-93f7-4322-8829-3d50d7496c5f (at 192.168.204.6@tcp) reconnecting [ 5118.590325] Lustre: 139610:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5122.420584] Lustre: *** cfs_fail_loc=159, val=0*** [ 5122.423440] Lustre: Skipped 4 previous similar messages [ 5138.060706] Lustre: lustre-MDT0000: Client 50f2f38a-04ac-4853-85d0-f9d3df10ff8c (at 192.168.204.6@tcp) reconnecting [ 5140.701055] Lustre: *** cfs_fail_loc=159, val=0*** [ 5140.734017] Lustre: Skipped 14 previous similar messages [ 5156.557609] Lustre: lustre-MDT0000: Client 50f2f38a-04ac-4853-85d0-f9d3df10ff8c (at 192.168.204.6@tcp) reconnecting [ 5158.620210] Lustre: 139898:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5166.564391] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5166.570125] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5166.596680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5168.481438] Lustre: server umount lustre-OST0000 complete [ 5174.759637] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5174.761652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5174.769207] Lustre: Skipped 1 previous similar message [ 5174.794240] Lustre: Skipped 1 previous similar message [ 5179.873796] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5179.884374] Lustre: Skipped 1 previous similar message [ 5183.975048] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5183.986809] Lustre: Skipped 1 previous similar message [ 5184.091726] Lustre: server umount lustre-MDT0000 complete [ 5189.097484] LustreError: 139848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5189.129888] LustreError: 139848:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5189.644289] LustreError: 136545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786190793 with bad export cookie 12982003298373629892 [ 5189.662124] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5189.662445] LustreError: 136545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5190.078039] Lustre: server umount lustre-MDT0001 complete [ 5208.353150] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5212.008975] Key type lgssc unregistered [ 5212.245956] LNet: 140979:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5212.252725] LNetError: 140979:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5212.266949] LNet: Removed LNI 192.168.204.106@tcp [ 5213.456247] Key type .llcrypt unregistered [ 5213.461076] Key type ._llcrypt unregistered [ 5250.871191] Key type ._llcrypt registered [ 5250.873465] Key type .llcrypt registered [ 5251.002299] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5252.342086] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5252.618227] alg: No test for adler32 (adler32-zlib) [ 5255.188712] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 5255.699868] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 5257.521631] Key type lgssc registered [ 5259.399637] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5272.808661] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5272.843291] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5274.395969] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5280.370359] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5283.812203] LustreError: 142410:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5283.837383] LustreError: 142410:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5288.936926] LustreError: 142411:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5292.470512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5292.864347] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5299.985767] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5311.661926] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5319.698945] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5329.693925] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5330.117665] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5337.957206] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5338.348731] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5344.457332] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5350.461917] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5358.714976] Lustre: 144999:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5363.588499] Lustre: 145048:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5365.686675] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5366.791945] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5372.906355] Lustre: 145194:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5374.369716] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5376.347925] Lustre: 145303:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5384.674420] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5384.684733] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5384.704821] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5385.703630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5385.711640] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5385.743776] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5388.464991] Lustre: server umount lustre-OST0000 complete [ 5395.426701] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5395.439305] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5395.463059] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5395.472794] Lustre: Skipped 1 previous similar message [ 5399.712754] Lustre: server umount lustre-MDT0000 complete [ 5401.058807] LustreError: 143332:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5401.086241] LustreError: 143332:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5404.371597] LustreError: 142392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191008 with bad export cookie 8489315871880026801 [ 5404.376424] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5404.407153] LustreError: 142392:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5404.857303] Lustre: server umount lustre-MDT0001 complete [ 5422.204697] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5426.179041] Key type lgssc unregistered [ 5426.693852] LNet: 146379:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5426.707661] LNetError: 146379:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5427.751890] LNet: Removed LNI 192.168.204.106@tcp [ 5429.046399] Key type .llcrypt unregistered [ 5429.054457] Key type ._llcrypt unregistered [ 5465.109283] Key type ._llcrypt registered [ 5465.112912] Key type .llcrypt registered [ 5465.257323] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5466.831868] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5466.852197] alg: No test for adler32 (adler32-zlib) [ 5468.604170] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 5468.960372] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 5470.824866] Key type lgssc registered [ 5473.093816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5483.455926] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5483.482707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5484.963721] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5490.690627] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5495.271595] LustreError: 147798:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5495.300730] LustreError: 147798:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5500.395689] LustreError: 147799:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5501.392095] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5501.910106] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5508.070336] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5519.450775] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5527.113710] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5536.034073] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5536.478893] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5543.192328] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5546.765032] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5554.118520] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5554.924924] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5562.430238] Lustre: *** cfs_fail_loc=159, val=0*** [ 5578.930683] Lustre: lustre-MDT0000: Client 58867735-0320-4c6a-b78c-2cfd5c7c7253 (at 192.168.204.6@tcp) reconnecting [ 5584.871802] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 5584.874043] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5584.888873] LustreError: Skipped 1 previous similar message [ 5584.906052] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5585.915889] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5585.917743] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5590.872659] Lustre: server umount lustre-OST0000 complete [ 5596.134829] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5596.146888] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5596.153917] Lustre: Skipped 2 previous similar messages [ 5601.252691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5601.261389] Lustre: Skipped 1 previous similar message [ 5602.271573] Lustre: server umount lustre-MDT0000 complete [ 5606.381954] LustreError: 147794:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5606.420842] LustreError: 147794:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5607.151917] LustreError: 147778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191211 with bad export cookie 18303123946611949480 [ 5607.160596] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5607.166858] LustreError: 147778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5607.510916] Lustre: server umount lustre-MDT0001 complete [ 5618.528257] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5621.250767] Key type lgssc unregistered [ 5621.621394] LNet: 151464:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5621.628844] LNetError: 151464:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5622.694449] LNet: Removed LNI 192.168.204.106@tcp [ 5624.101451] Key type .llcrypt unregistered [ 5624.104426] Key type ._llcrypt unregistered [ 5654.048237] Key type ._llcrypt registered [ 5654.050243] Key type .llcrypt registered [ 5654.329571] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5655.765413] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5655.800952] alg: No test for adler32 (adler32-zlib) [ 5657.193647] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 5657.421473] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 5659.175217] Key type lgssc registered [ 5660.780077] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5670.048848] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5670.071601] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5671.441736] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5675.884406] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5681.639311] LustreError: 152880:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5681.664589] LustreError: 152880:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5685.602636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5686.070845] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5691.677719] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5699.832421] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5705.921433] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5714.366873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5714.756425] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5715.836654] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5721.541863] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5729.684434] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5731.187826] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5749.657688] Lustre: lustre-MDT0001: evicting 035c34b4-354a-4d81-811d-a1d04db03444 (at 192.168.204.6@tcp) by administrative request [ 5749.666164] Lustre: Skipped 1 previous similar message [ 5782.506197] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5782.527647] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5782.558885] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5782.579468] Lustre: Skipped 1 previous similar message [ 5787.617900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5792.746503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5792.752267] Lustre: Skipped 2 previous similar messages [ 5793.247173] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5793.442745] Lustre: server umount lustre-OST0000 complete [ 5797.867775] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5797.879939] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5797.891950] Lustre: Skipped 2 previous similar messages [ 5797.909618] Lustre: Skipped 1 previous similar message [ 5802.979935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5802.987407] Lustre: Skipped 1 previous similar message [ 5804.212947] Lustre: server umount lustre-MDT0000 complete [ 5808.097917] LustreError: 152881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5808.117121] LustreError: 152881:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5808.168207] LustreError: 152862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191412 with bad export cookie 18331139902747409505 [ 5808.170807] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5808.180023] LustreError: 152862:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5808.598860] Lustre: server umount lustre-MDT0001 complete [ 5822.106941] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5825.027351] Key type lgssc unregistered [ 5825.285706] LNet: 156958:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5825.297572] LNetError: 156958:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5825.313909] LNet: Removed LNI 192.168.204.106@tcp [ 5826.017473] Key type .llcrypt unregistered [ 5826.022339] Key type ._llcrypt unregistered [ 5895.617303] Key type ._llcrypt registered [ 5895.620784] Key type .llcrypt registered [ 5895.730340] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 5896.419580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5896.495876] alg: No test for adler32 (adler32-zlib) [ 5897.508907] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 5897.682433] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 5899.359207] Key type lgssc registered [ 5900.721503] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5907.423959] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5907.446542] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5908.774304] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5913.132693] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5919.202526] LustreError: 159426:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5919.227647] LustreError: 159426:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5921.474732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5921.839382] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5926.131625] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5932.856567] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5938.352694] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5944.962452] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5945.353810] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5947.491518] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5951.479401] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 5954.612967] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5959.040921] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5967.844456] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5967.856196] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5967.865667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5969.891250] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5969.892872] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5969.909373] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5973.367553] Lustre: server umount lustre-OST0000 complete [ 5978.082405] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5978.086593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5978.097693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5978.104677] Lustre: Skipped 1 previous similar message [ 5980.134025] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5980.135721] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5982.752709] Lustre: server umount lustre-MDT0000 complete [ 5985.248927] LustreError: 159426:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5985.264355] LustreError: 159426:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5985.834108] LustreError: 159406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191590 with bad export cookie 9800919169325846973 [ 5985.835579] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5985.840028] LustreError: 159406:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5986.086963] Lustre: server umount lustre-MDT0001 complete [ 5994.646737] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 5996.863530] Key type lgssc unregistered [ 5997.164391] LNet: 162968:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5997.175976] LNetError: 162968:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5997.197597] LNet: Removed LNI 192.168.204.106@tcp [ 5997.935646] Key type .llcrypt unregistered [ 5997.937964] Key type ._llcrypt unregistered [ 6019.711572] Key type ._llcrypt registered [ 6019.715773] Key type .llcrypt registered [ 6019.857613] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 6020.805379] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6020.864788] alg: No test for adler32 (adler32-zlib) [ 6021.912906] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 6022.190951] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 6023.880109] Key type lgssc registered [ 6024.940250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6037.748797] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 08:20:40 (1786191640) [ 6052.749847] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 6061.070782] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 6094.740126] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6094.776512] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6096.041610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6096.084833] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6096.187501] Lustre: lustre-MDT0000: new disk, initializing [ 6096.272490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6096.289772] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6100.517463] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6109.709279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6109.816231] Lustre: 167233:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6109.883593] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6109.889962] Lustre: Skipped 1 previous similar message [ 6109.972783] Lustre: lustre-MDT0001: new disk, initializing [ 6110.054782] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6110.089323] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6110.104269] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6113.738487] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6121.413277] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6126.968469] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6132.793316] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6132.991543] Lustre: lustre-OST0000: new disk, initializing [ 6132.997584] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6133.002739] Lustre: 169199:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6133.068471] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6138.164552] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6138.450378] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6138.465892] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6138.548743] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6145.808545] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6154.209566] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6154.215895] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6154.238993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6154.257510] Lustre: Skipped 1 previous similar message [ 6158.551191] Lustre: server umount lustre-OST0000 complete [ 6164.450843] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6164.458260] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6164.472068] Lustre: Skipped 2 previous similar messages [ 6167.671437] Lustre: server umount lustre-MDT0000 complete [ 6170.383167] LustreError: 168025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786191774 with bad export cookie 14474269842217830967 [ 6170.386790] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6170.391188] LustreError: 168025:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6170.596903] Lustre: server umount lustre-MDT0001 complete [ 6178.560883] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 6180.787435] Key type lgssc unregistered [ 6181.039954] LNet: 171107:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6181.051314] LNetError: 171107:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6181.063107] LNet: Removed LNI 192.168.204.106@tcp [ 6181.659922] Key type .llcrypt unregistered [ 6181.662997] Key type ._llcrypt unregistered [ 6199.691541] Key type ._llcrypt registered [ 6199.694212] Key type .llcrypt registered [ 6199.796762] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 6200.443240] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6200.506308] alg: No test for adler32 (adler32-zlib) [ 6201.503321] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 6201.694575] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 6203.327267] Key type lgssc registered [ 6204.256827] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6211.344954] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6211.370265] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6212.722924] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6216.104621] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6223.333361] LustreError: 172501:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6223.349283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6223.364345] LustreError: 172501:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6223.638774] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6227.262335] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6233.734541] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6238.366796] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6243.458873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6243.703613] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6248.257829] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 6254.226954] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6258.043612] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6264.085464] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6264.093949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6264.106504] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6264.114604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.123946] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6264.868173] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6264.893364] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6264.906768] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6264.913855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.922734] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6265.620698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.152798] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6266.156770] Lustre: Skipped 1 previous similar message [ 6266.165990] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6266.173216] Lustre: Skipped 1 previous similar message [ 6266.176143] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6266.178532] Lustre: Skipped 1 previous similar message [ 6266.184299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.190960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6266.195931] Lustre: Skipped 1 previous similar message [ 6266.915358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.608599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.323920] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6268.328600] Lustre: Skipped 2 previous similar messages [ 6268.350898] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6268.365119] Lustre: Skipped 2 previous similar messages [ 6268.369592] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6268.373131] Lustre: Skipped 2 previous similar messages [ 6268.376356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.389310] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6268.392331] Lustre: Skipped 2 previous similar messages [ 6269.079634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.783878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.436754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.220620] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6271.229209] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6271.240527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.927797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.733960] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6272.746112] Lustre: Skipped 5 previous similar messages [ 6272.766623] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6272.776516] Lustre: Skipped 5 previous similar messages [ 6272.780782] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6272.784560] Lustre: Skipped 5 previous similar messages [ 6272.790846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.801920] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6272.809126] Lustre: Skipped 5 previous similar messages [ 6273.464946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.254489] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6274.268171] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6274.277467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.955376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.668360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.284441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.999707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.582152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.209468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.904477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.546253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.254582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.012674] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6281.018870] Lustre: Skipped 11 previous similar messages [ 6281.038810] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6281.048427] Lustre: Skipped 11 previous similar messages [ 6281.050127] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6281.054846] Lustre: Skipped 11 previous similar messages [ 6281.057596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.068211] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6281.074215] Lustre: Skipped 11 previous similar messages [ 6281.708923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.456966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.184900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.939714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.729400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.544287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.235410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.019231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.682445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.555240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.260850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.136942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.756775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.632776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.362124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.296193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.048072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.009932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.733596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.653396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.379854] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6297.388059] Lustre: Skipped 20 previous similar messages [ 6297.404812] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6297.411796] Lustre: Skipped 20 previous similar messages [ 6297.417275] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6297.420740] Lustre: Skipped 20 previous similar messages [ 6297.424638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.440812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6297.450687] Lustre: Skipped 20 previous similar messages [ 6298.141395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.831605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.570720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.164361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.914150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.573710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.444966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.148865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.920252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.571485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.162103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.770751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.389458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.017857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.689189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.347340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.975945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.582562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.260588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.927902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.673353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.286093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.057411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.701308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.460939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.034695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.633182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.186818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.843453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.515891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.175504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.849487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.527468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.096526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.690885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.204844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.752972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.263661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.818032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.438976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.134047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.738805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.377554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.973960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.739368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.392584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.938388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.494616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.248076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.846518] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6329.851215] Lustre: Skipped 49 previous similar messages [ 6329.860277] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6329.866159] Lustre: Skipped 49 previous similar messages [ 6329.869405] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6329.872886] Lustre: Skipped 49 previous similar messages [ 6329.875595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.881884] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6329.885848] Lustre: Skipped 49 previous similar messages [ 6330.646220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.570933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.407230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.136722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.908688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.505869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.144617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.740841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.388069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.008431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.676219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.302964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.905922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.485799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.177466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.758867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.373758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.992927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.742829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.350237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.007056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.727196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.498611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.122282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.773962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.371324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.989363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.545357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.124857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.690598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.336792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.925363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.549094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.213511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.839567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.431096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.003666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.594744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.143326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.665453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.193645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.804262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.374594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.944204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.550568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.152778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.697233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.263381] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6360.270475] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6360.272644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.811311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.382116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.875189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.370187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.031485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.661408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.201216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.771274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.294764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.846876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.469741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.057186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.668902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.289091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.003889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.633689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.315673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.061087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.698274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.284991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.969057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.504357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.127285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.729904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.348764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.946485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.627196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.249666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.805727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.373665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.095241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.843214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.433793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.060265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.674156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.299196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.902384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.532945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.220710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.876086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.467357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.080308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.725514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.333277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.968475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.611591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.245323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.843913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.467269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.124537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.709508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.405064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.022201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.827097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.438802] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6394.441798] Lustre: Skipped 102 previous similar messages [ 6394.450282] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6394.459869] Lustre: Skipped 102 previous similar messages [ 6394.463962] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6394.468789] Lustre: Skipped 102 previous similar messages [ 6394.472346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.478475] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6394.482820] Lustre: Skipped 102 previous similar messages [ 6395.082677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.898290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.556744] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6396.564719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.566679] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6397.311695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.020847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.642667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.281711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.907472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.559524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.267912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.869324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.550470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.202172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.804166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.426419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.003241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.568064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.107658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.659968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.177786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.761104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.352535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.950896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.522204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.149980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.755331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.382820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.991805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.628565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.196610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.858871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.469259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.147059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.759361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.430212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.985156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.655493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.286154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.012326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.651849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.309306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.973694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.738851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.316541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.046334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.664709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.328014] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.971319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.608269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.170034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.709351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.230150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.771865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.359745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.972583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.499705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.026410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.564550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.065849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.594861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.201225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.768270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.248436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.786691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.344424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.910317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.529699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.125946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.782967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.441629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.050838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.641868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.245749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.888361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.486618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.144180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.817392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.430413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.042459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.622372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.260219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.906401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.506084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.100532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.704136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.235825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.794599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.270286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.732405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.271289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.797668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.353811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.916163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.564900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.147249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.774845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.450990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.050666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.564190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.128687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.732549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.308761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.858268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.491416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.097644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.717564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.263821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.872889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.434703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.967426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.592593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.143552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.733611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.308305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.865620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.414392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.986453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.535260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.055754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.628083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.162333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.808786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.376426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.041398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.572785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.222817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.818180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.421531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.000079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.607511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.234612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.831619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.447911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.034583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.618767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.194021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.744685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.272239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.778744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.332281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.892895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.478119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.052916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.585603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.104837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.713370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.295266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.902824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.540459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.129574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.696403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.313079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.897082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.519502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.176094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.834373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.537770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.307057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.051192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.856709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.527653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.423356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.057127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.661691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.305507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.942937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.520767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.087634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.702249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.316944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.926152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.616550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.190635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.840405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.460397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.127722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.710969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.369440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.972572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.729490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.307045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.881855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.458403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.091367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.713326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.312630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.950465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.505505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.063773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.664833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.168869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.764495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.336302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.009207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.551461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.126877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.680902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.241423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.807447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.452525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.038333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.753092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.366979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.016518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.594615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.244593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.911518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.473769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.097265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.708528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.294421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.035303] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6523.040483] Lustre: Skipped 211 previous similar messages [ 6523.061325] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6523.073215] Lustre: Skipped 211 previous similar messages [ 6523.078699] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6523.082370] Lustre: Skipped 211 previous similar messages [ 6523.087047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.095549] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6523.102625] Lustre: Skipped 211 previous similar messages [ 6523.692249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.336823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.090673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.852927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.552709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.303665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.945922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.794550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.479519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.164818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.760073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.417192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.961095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.564211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.176551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.788299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.368333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.945830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.464115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.984368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.572704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.121110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.696168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.288995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.889890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.428562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.989720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.595598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.200271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.739634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.302798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.851756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.384224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.917362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.390978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.948944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.451771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.995265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.514326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.086712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.600841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.142611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.689844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.208763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.712559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.246821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.804700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.361759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.877803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.423897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.959940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.497322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.984201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.524924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.991086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.597714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.109231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.649661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.268782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.833772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.350578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.873880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.356382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.942576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.504709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.041198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.576685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.041940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.631745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.176311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.741900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.259829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.819774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.327658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.963328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.507948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.061613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.619936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.243750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.874772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.556856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.120155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.781817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.382601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.961427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.543151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.189970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.729410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.321558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.929303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.430085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.953857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.571441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.105402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.720366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.315868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.904133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.469486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.000392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.512668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.067705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.561739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.114728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.574228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.093135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.725141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.265270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.779110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.277824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.755047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.260532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6586.740361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.229978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.757132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.256739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.744945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.236988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.723413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.230954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.761227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.212799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.692943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.159110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.635164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.124945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.628340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.082531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.565558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.051899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.526281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.013580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.572948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.034400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.525104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.083558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.509591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.037036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.533354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.972365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.430102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.918771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.366365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.789198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.291042] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6602.298080] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6602.300066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.762136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.210149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.717419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.178083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.629244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.161474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.641881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.150781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.692533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.161745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.673606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.159424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.640964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.095912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.564749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.003734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.417318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.877208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.358434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.881290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.422274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.918158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.469674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.994679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.533574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.208077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.725222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.250132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.850915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.426457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.000096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.560261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.189420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.775601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.398188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.014404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.620716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.260206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.893082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.432519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.027405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.602451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.205689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.766584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.330191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.890102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.460716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.022593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.567825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.065725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.560690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.045146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.488272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.959302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.460211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.918774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.421921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.973219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.477770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.027856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.547432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.979894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.424902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.875940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.299231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.781862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.256698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.777279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.279964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.753877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.228538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.705248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.149863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.604687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.122725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.640831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.180857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.774330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.349802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.842994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.350943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.887161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.413380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.895675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.505593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.061967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.587351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.076864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.588960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.114067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.647606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.195566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.734113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.271748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.807270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.296724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.795082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.378064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.918654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.381539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.841259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.405407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.952463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.516935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.037191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.562697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.166093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.677198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.211273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.733511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.264086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.764536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.290082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.793365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.359666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.941639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.511677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.118779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.654717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.232083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6665.800478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.342366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.843793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.374668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.942954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.612348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.210480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.752279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.286843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.843448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.400245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.979693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.519354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.048595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.626941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.207193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.783868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.318205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.852384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.387792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.894843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.521262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.023337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.512789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.056195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6679.613192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.142091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.681278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.282498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.832025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.333071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.901497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.468161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.059505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.657712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.267233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.851470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.395560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.019352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.572961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.098826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6688.615438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.184698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.709033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.216243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.779022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.359977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.883236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.430387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.960517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.458819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.111875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.591088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.177594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.740422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.297494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.929116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6697.486919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.009563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.530493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.042397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.587530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.140502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.700689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.173056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6701.742931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.295829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.798206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.262520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.764040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.269485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.766709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.338440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6705.827772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.314975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.889961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.431114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.982699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.420453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.891738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.398717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.944608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.466245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.030333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.578938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.125947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.667907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.217903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.733341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.223269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.739544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.255924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6715.769195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.302869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.836972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.453447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.078118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.577701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.021427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.535805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6719.990499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.437668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.909838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.467492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.020362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.574255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.156272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6723.639911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.114019] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.656438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.194751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.760510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.323582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.927593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.490304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.105741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6728.628947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.114983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.611150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.148147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.675558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.169126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6731.716219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.314109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.825955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.394908] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6733.404239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.405729] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6733.905756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.491911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.020645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.607220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.125334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.637804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.266409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6737.839552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.368339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.928304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.509365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.082211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.612064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.156980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.704100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.242182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.848715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.438247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.983406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.588792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.132740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.653933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.191822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.781767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.334872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.876771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.428955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.970452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.456833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.957213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.470285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.962325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.405865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.947996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.419914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.868560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.389553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6753.846110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.373502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.930891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.418937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.843432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.351337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.868211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.360335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6757.874859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.354151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.826320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.319610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.818332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.327495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.849391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.336262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.860455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.336593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.784100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.257976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.751026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.234936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.737523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.261393] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6765.266907] LustreError: 172520:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6765.269702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6765.726687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.207973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.686492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.160130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6767.603832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.144612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.640863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.073328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.550877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.974621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.423566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.883945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.360836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.903166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.475325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.976615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.492477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.996463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.455831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.995229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.573489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.093515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.701443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.273146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.925064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.519589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.105470] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6779.111361] Lustre: Skipped 477 previous similar messages [ 6779.129387] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6779.138207] Lustre: Skipped 477 previous similar messages [ 6779.141637] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6779.146369] Lustre: Skipped 477 previous similar messages [ 6779.149606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6779.155593] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6779.161015] Lustre: Skipped 477 previous similar messages [ 6779.737464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.320094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.875286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.376267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.917349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.473756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6782.969319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.526568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.057620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6784.595396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.113825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.727350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.306624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.774878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.264652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.774554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.250319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.739293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.272842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.808702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.327225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.822915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.323902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.883262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.382965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.854023] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.449506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.928255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.428547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.005474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.463371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.042629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.668068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.263099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.906764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.456943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.142982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.695740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.419432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.955416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.589721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.133888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.684947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.178975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.700574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.256656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.781933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.313775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.793661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.299260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.885759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.483571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.025808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.561468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.154532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.711704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.392897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.934905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.486626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.953551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.513246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.052758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.563152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.121588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.719163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.259964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6815.806903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.388200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.932090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.420520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.903255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.425479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.921802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.412573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.884837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.351578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.781297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.228244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.700589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.187223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.705171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.206408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.687746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.270467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.785261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.281647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6825.800791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.261316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.698782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.139032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.558961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.998100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.476238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.983575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.536128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.056755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.516059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.075801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.552618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.060449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.594822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.130607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.602320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.091275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.560326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.028516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6835.542786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.027928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.557975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.069388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.644157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.888983] LustreError: 174321:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7167.895387] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7167.900155] LustreError: 172520:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 7364.066969] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7364.068167] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7364.071700] Lustre: Skipped 111 previous similar messages [ 7364.072924] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7364.082628] Lustre: Skipped 1 previous similar message [ 7367.961167] Lustre: server umount lustre-OST0000 complete [ 7370.719871] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7370.723149] LustreError: Skipped 1 previous similar message [ 7370.726491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7374.305064] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7374.307480] Lustre: Skipped 1 previous similar message [ 7375.999488] Lustre: server umount lustre-MDT0000 complete [ 7377.564427] LustreError: 172482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786192981 with bad export cookie 1735731080671548174 [ 7377.567159] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7377.571928] LustreError: 172482:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7377.720142] Lustre: server umount lustre-MDT0001 complete [ 7382.259836] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 7383.555462] Key type lgssc unregistered [ 7383.701514] LNet: 227615:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7383.706371] LNetError: 227615:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7383.713452] LNet: Removed LNI 192.168.204.106@tcp [ 7384.114549] Key type .llcrypt unregistered [ 7384.116409] Key type ._llcrypt unregistered [ 7396.731804] Key type ._llcrypt registered [ 7396.733291] Key type .llcrypt registered [ 7396.777904] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7397.296884] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7397.323060] alg: No test for adler32 (adler32-zlib) [ 7398.228988] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 7398.336996] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 7399.935296] Key type lgssc registered [ 7400.418823] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7404.243714] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7404.253870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7405.456831] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7407.320734] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7411.494871] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7411.707916] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7413.595129] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7417.119706] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7419.719776] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7422.748089] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7422.926218] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7425.497630] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7428.640858] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7430.126124] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7441.088103] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7441.791170] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7441.797956] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7441.803174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.808334] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7441.808366] LustreError: 229026:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7445.410263] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7445.412227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7445.414887] Lustre: Skipped 1 previous similar message [ 7448.699163] Lustre: server umount lustre-OST0000 complete [ 7461.887222] LustreError: 229026:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7478.331377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7478.513940] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7481.351455] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7485.019553] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7485.669280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7485.677903] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7485.678977] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7485.685437] Lustre: Skipped 1 previous similar message [ 7486.791152] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 1 sec [ 7489.483134] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c9e0b925000.ost_server_uuid 50 [ 7490.274172] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c9e0b925000.ost_server_uuid in FULL state after 0 sec [ 7496.162912] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7496.167329] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7496.169606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7496.175965] Lustre: Skipped 1 previous similar message [ 7496.182994] Lustre: Skipped 3 previous similar messages [ 7498.076764] Lustre: server umount lustre-OST0000 complete [ 7501.281656] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7501.282572] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7501.286936] Lustre: Skipped 1 previous similar message [ 7506.401339] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7506.404618] Lustre: Skipped 2 previous similar messages [ 7511.520750] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7511.523394] Lustre: Skipped 1 previous similar message [ 7514.079192] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7514.161728] Lustre: server umount lustre-MDT0000 complete [ 7515.795275] LustreError: 232256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193120 with bad export cookie 2956319765676930121 [ 7515.798183] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7515.798903] LustreError: 232256:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7515.935322] Lustre: server umount lustre-MDT0001 complete [ 7520.538634] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 7521.691656] Key type lgssc unregistered [ 7521.843524] LNet: 234239:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7521.846526] LNetError: 234239:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7521.855419] LNet: Removed LNI 192.168.204.106@tcp [ 7522.199532] Key type .llcrypt unregistered [ 7522.201108] Key type ._llcrypt unregistered [ 7537.041067] Key type ._llcrypt registered [ 7537.042589] Key type .llcrypt registered [ 7537.087947] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7537.449655] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7537.487063] alg: No test for adler32 (adler32-zlib) [ 7538.379636] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 7538.493445] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 7540.095228] Key type lgssc registered [ 7540.567088] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7544.622558] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7544.631876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7545.812026] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7547.471755] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7551.091984] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7551.218092] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7552.731759] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7553.844138] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7556.481063] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7556.635948] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7559.051493] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7562.940112] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7563.021425] Lustre: lustre-OST0001: new disk, initializing [ 7563.024149] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7563.027738] Lustre: 238249:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7563.067742] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7565.471935] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7567.873462] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7567.877429] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7567.903828] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7570.983933] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7575.530203] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7578.304598] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7586.274435] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7586.277052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7586.280047] Lustre: Skipped 3 previous similar messages [ 7586.284526] Lustre: Skipped 3 previous similar messages [ 7591.393823] LustreError: 235943:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7591.399626] LustreError: 235943:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 7591.413617] Lustre: server umount lustre-MDT0000 complete [ 7592.789319] LustreError: 235924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193197 with bad export cookie 3796637834863310112 [ 7592.791962] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7592.793970] LustreError: 235924:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7592.916615] Lustre: server umount lustre-MDT0001 complete [ 7603.722525] Lustre: server umount lustre-OST0000 complete [ 7615.476082] Lustre: server umount lustre-OST0001 complete [ 7623.226584] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7626.890502] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7627.039727] LustreError: 241353:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7627.058143] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7628.624280] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7631.737928] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7633.271614] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7634.373909] Lustre: 242495:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7636.794833] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7636.933221] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7636.936149] Lustre: Skipped 1 previous similar message [ 7637.989516] LustreError: 242848:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7637.994366] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7638.939189] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7642.084299] LustreError: 243031:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7642.092980] LustreError: 243031:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7642.178816] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7644.846493] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7647.722121] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7648.289299] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7649.558747] Lustre: 244362:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7656.231821] Lustre: server umount mylustre-MDT0000 complete [ 7657.440787] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7657.444406] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7657.450467] LustreError: 241353:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7657.685484] LustreError: 241328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193262 with bad export cookie 3796637834863312905 [ 7657.689894] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7657.692087] LustreError: 241328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7657.955365] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7657.960433] Lustre: Skipped 1 previous similar message [ 7659.843845] Lustre: server umount mylustre-MDT0001 complete [ 7661.406892] Lustre: server umount mylustre-OST0000 complete [ 7663.176843] Lustre: server umount mylustre-OST0001 complete [ 7671.723033] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7675.758899] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7675.937125] LustreError: 246661:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7675.947334] LustreError: 246661:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7675.971768] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7675.974774] Lustre: Skipped 1 previous similar message [ 7677.502122] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7680.679422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7682.265190] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7683.374644] Lustre: 247805:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7685.957082] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7688.363931] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7691.805922] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7694.031238] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7701.482790] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7701.482840] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7707.938566] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7709.373921] Lustre: 249694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7711.713125] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7711.716500] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7711.716973] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7711.717773] Lustre: Skipped 6 previous similar messages [ 7716.836642] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7716.839662] Lustre: Skipped 6 previous similar messages [ 7717.602962] Lustre: server umount tfs-MDT0000 complete [ 7718.977251] LustreError: 249697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193323 with bad export cookie 3796637834863314795 [ 7718.980603] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7718.981979] LustreError: 249697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7719.109225] Lustre: server umount tfs-MDT0001 complete [ 7730.182263] Lustre: server umount tfs-OST0000 complete [ 7741.731216] Lustre: server umount tfs-OST0001 complete [ 7749.393971] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7753.104443] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7753.242654] LustreError: 251956:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7753.249919] LustreError: 251956:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7753.261649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7753.264396] Lustre: Skipped 3 previous similar messages [ 7754.613150] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7757.731199] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7759.428635] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7760.514892] Lustre: 253096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7763.210637] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7764.457766] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7765.766278] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7769.080042] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7771.465843] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7774.192377] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7774.891597] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7776.174409] Lustre: 254964:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7778.952815] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:49:42 (1786193382) [ 7793.631960] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7793.634517] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7793.638197] Lustre: Skipped 1 previous similar message [ 7793.640133] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7797.082661] Lustre: server umount lustre-MDT0000 complete [ 7798.341500] LustreError: 254966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193402 with bad export cookie 3796637834863316454 [ 7798.344436] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7798.347291] LustreError: 254966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7799.776518] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7799.777048] LustreError: 255203:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7799.783851] Lustre: Skipped 4 previous similar messages [ 7799.790340] LustreError: 255203:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7803.808415] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7803.810638] Lustre: Skipped 7 previous similar messages [ 7804.537612] Lustre: server umount lustre-MDT0001 complete [ 7820.255132] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7820.295864] Lustre: server umount lustre-OST0000 complete [ 7827.674189] Lustre: server umount lustre-OST0001 complete [ 7829.916338] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 7833.185147] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 7845.950141] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7845.950141] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7845.950141] [ 7845.964890] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7846.046155] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7846.056896] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7846.094255] Lustre: lustre-MDT0000: new disk, initializing [ 7846.118937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7846.121721] Lustre: Skipped 3 previous similar messages [ 7846.127860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7847.627688] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7852.005555] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7852.005555] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7852.005555] [ 7852.014318] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7852.062590] Lustre: 258147:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7852.093242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7852.095721] Lustre: Skipped 1 previous similar message [ 7852.137706] Lustre: lustre-MDT0001: new disk, initializing [ 7852.186450] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7852.190787] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7853.574243] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7857.916949] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7858.028069] Lustre: lustre-OST0000: new disk, initializing [ 7858.030583] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7858.033224] Lustre: 259730:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7859.450569] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7859.454525] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7859.487540] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7860.069111] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7864.499056] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7864.568419] Lustre: lustre-OST0001: new disk, initializing [ 7864.570646] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7864.574858] Lustre: 260599:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7865.794492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7865.799232] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7865.830527] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7866.677910] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 7871.217040] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 08:51:14 (1786193474) [ 7874.935469] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 08:51:18 (1786193478) [ 7890.400115] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7890.400211] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7890.403024] LustreError: Skipped 1 previous similar message [ 7890.407728] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7890.411243] Lustre: Skipped 1 previous similar message [ 7892.417583] Lustre: server umount lustre-OST0000 complete [ 7896.547225] LustreError: 260786:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7896.553217] LustreError: 260786:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7899.874712] Lustre: server umount lustre-MDT0000 complete [ 7901.166202] LustreError: 258942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193505 with bad export cookie 3796637834863318008 [ 7901.169188] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7901.170305] LustreError: 258942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7907.367749] Lustre: server umount lustre-MDT0001 complete [ 7910.817821] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 7981.493149] Lustre: server umount lustre-OST0001 complete [ 7983.767883] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 7990.295884] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8002.978275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8003.073703] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8003.084867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8003.123351] Lustre: lustre-MDT0000: new disk, initializing [ 8003.149251] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8003.152836] Lustre: Skipped 3 previous similar messages [ 8003.160642] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8004.638514] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8008.996968] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8009.041917] Lustre: 265376:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8009.159058] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8010.749650] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8014.807138] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8016.825637] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8019.214342] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8019.337246] Lustre: lustre-OST0000: new disk, initializing [ 8019.339342] Lustre: Skipped 1 previous similar message [ 8019.341766] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8019.346542] Lustre: Skipped 2 previous similar messages [ 8019.349983] Lustre: 267343:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8020.924796] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8020.928043] Lustre: Skipped 1 previous similar message [ 8020.930945] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8020.974479] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8021.531736] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8025.636598] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8031.202200] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8031.202849] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8031.205094] LustreError: Skipped 1 previous similar message [ 8031.208701] Lustre: Skipped 5 previous similar messages [ 8031.211040] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8031.214525] Lustre: Skipped 8 previous similar messages [ 8034.347080] Lustre: server umount lustre-OST0000 complete [ 8041.814188] Lustre: server umount lustre-MDT0000 complete [ 8043.082101] LustreError: 265369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193647 with bad export cookie 3796637834863319478 [ 8043.085104] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8043.086623] LustreError: 265369:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8043.211193] Lustre: server umount lustre-MDT0001 complete [ 8047.002740] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8048.100610] Key type lgssc unregistered [ 8048.234515] LNet: 269103:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8048.237966] LNetError: 269103:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8048.245482] LNet: Removed LNI 192.168.204.106@tcp [ 8048.594138] Key type .llcrypt unregistered [ 8048.595601] Key type ._llcrypt unregistered [ 8058.974302] Key type ._llcrypt registered [ 8058.975811] Key type .llcrypt registered [ 8059.014643] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8059.369117] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8059.414037] alg: No test for adler32 (adler32-zlib) [ 8060.277331] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 8060.373524] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8061.959255] Key type lgssc registered [ 8062.368595] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8065.219055] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8065.225672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8066.386214] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8067.831755] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8071.109069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8071.264376] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8072.794756] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8075.564356] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8077.708386] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8080.126377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8080.248111] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8082.280923] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8085.040967] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8085.398823] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8086.497691] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8086.497801] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8086.502221] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8088.145831] Lustre: server umount lustre-OST0000 complete [ 8091.616412] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8091.617200] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8091.625221] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8091.627892] Lustre: Skipped 1 previous similar message [ 8091.686454] Lustre: server umount lustre-MDT0000 complete [ 8093.019207] LustreError: 270524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193697 with bad export cookie 7136568517113158575 [ 8093.022230] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8093.023771] LustreError: 270524:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8099.183829] Lustre: server umount lustre-MDT0001 complete [ 8103.023617] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8104.167208] Key type lgssc unregistered [ 8104.306519] LNet: 274124:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8104.309714] LNetError: 274124:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8104.318369] LNet: Removed LNI 192.168.204.106@tcp [ 8104.644633] Key type .llcrypt unregistered [ 8104.646250] Key type ._llcrypt unregistered [ 8111.629896] Key type ._llcrypt registered [ 8111.631109] Key type .llcrypt registered [ 8111.670288] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8111.978377] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8111.984211] alg: No test for adler32 (adler32-zlib) [ 8112.846761] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 8112.938704] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8114.519167] Key type lgssc registered [ 8114.951454] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8118.273918] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8118.281936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8133.727295] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8138.847249] LustreError: 275296:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.106@tcp: failed processing log, type 4: rc = -110 [ 8165.471263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8167.568754] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8170.218875] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8171.034313] Lustre: server umount lustre-OST0000 complete [ 8176.895481] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8178.001027] Key type lgssc unregistered [ 8178.140488] LNet: 276929:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8178.143949] LNetError: 276929:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8178.151352] LNet: Removed LNI 192.168.204.106@tcp [ 8178.453113] Key type .llcrypt unregistered [ 8178.454658] Key type ._llcrypt unregistered [ 8192.819898] Key type ._llcrypt registered [ 8192.821088] Key type .llcrypt registered [ 8192.863724] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8193.223881] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8193.267673] alg: No test for adler32 (adler32-zlib) [ 8194.130405] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 8194.230035] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8195.815131] Key type lgssc registered [ 8196.184295] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8200.769053] loop0: detected capacity change from 0 to 204800000 [ 8200.829316] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8200.833328] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8200.957517] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8201.331479] loop0: detected capacity change from 0 to 204800000 [ 8201.427656] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8201.432148] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8201.559195] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8201.925170] loop0: detected capacity change from 0 to 204800000 [ 8202.037197] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8202.041210] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8202.084350] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8202.454861] loop0: detected capacity change from 0 to 204800000 [ 8202.514110] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8202.903789] loop0: detected capacity change from 0 to 204800000 [ 8202.913796] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8204.307527] loop0: detected capacity change from 0 to 204800000 [ 8204.316432] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8205.693382] loop0: detected capacity change from 0 to 204800000 [ 8205.711066] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8207.081598] loop0: detected capacity change from 0 to 204800000 [ 8207.093179] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8208.456982] loop0: detected capacity change from 0 to 204800000 [ 8208.497600] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8208.509860] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8208.543560] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8208.547617] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8208.552223] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8208.557093] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8208.561587] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8208.566318] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8208.570855] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8208.575665] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8208.580809] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8208.586349] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8208.590655] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8208.595510] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8208.600842] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8208.604638] Lustre: 280325:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8211.356894] Lustre: server umount MGS complete [ 8211.667457] loop0: detected capacity change from 0 to 204800000 [ 8211.696853] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8211.844353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8211.860723] Lustre: lustre-MDT0000: Aborting client recovery [ 8212.183284] loop1: detected capacity change from 0 to 204800000 [ 8212.215406] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8212.260686] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8212.265495] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8212.270594] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8212.275490] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8212.279587] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8212.284510] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8212.288958] Lustre: 280759:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8212.297939] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8212.362046] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8212.384282] Lustre: lustre-MDT0001: Aborting client recovery [ 8212.712057] loop2: detected capacity change from 0 to 204800000 [ 8212.771308] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8212.794365] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8212.800092] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8212.805486] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8212.811265] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8212.815283] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8212.820118] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8212.823924] Lustre: 280841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8212.869325] Lustre: Found index 0 for lustre-OST0000, updating log [ 8212.882355] Lustre: 280859:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8212.907359] Lustre: lustre-OST0000: Aborting recovery [ 8213.234898] loop3: detected capacity change from 0 to 204800000 [ 8213.273802] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8213.306212] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8213.312745] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8213.317687] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8213.322733] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8213.328078] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8213.332213] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8213.337475] Lustre: 280924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8213.375527] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8213.377600] Lustre: Skipped 1 previous similar message [ 8218.596883] LustreError: 280703:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 6, retries 0, failed: rc = -108 [ 8222.689543] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8222.689855] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8222.690461] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8222.695523] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8222.702785] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8222.709060] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8223.809032] LustreError: 280668:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 295us: rc = -2 [ 8238.047994] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8238.052306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8238.054810] Lustre: Skipped 3 previous similar messages [ 8241.190771] Lustre: server umount lustre-MDT0000 complete [ 8241.522528] LustreError: 280766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786193845 with bad export cookie 106483284959543856 [ 8241.524294] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8241.526755] LustreError: 280766:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8243.167949] LustreError: 281168:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8243.168074] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8243.168854] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8243.174172] LustreError: 281168:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8243.178761] Lustre: Skipped 4 previous similar messages [ 8246.175518] LustreError: 281161:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8246.175566] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8246.180132] LustreError: 281161:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8246.181940] Lustre: Skipped 2 previous similar messages [ 8247.788790] Lustre: server umount lustre-MDT0001 complete [ 8254.208932] Lustre: server umount lustre-OST0000 complete [ 8260.625257] Lustre: server umount lustre-OST0001 complete [ 8263.227973] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 08:57:46 (1786193866) [ 8272.509877] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8275.536808] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8286.909063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8286.988623] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8287.001403] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8287.037224] Lustre: lustre-MDT0000: new disk, initializing [ 8287.057584] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8287.063435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8288.378743] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8292.541801] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8292.587095] Lustre: 284989:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8292.611097] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8292.613405] Lustre: Skipped 1 previous similar message [ 8292.648674] Lustre: lustre-MDT0001: new disk, initializing [ 8292.682591] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8292.694593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8292.697699] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8294.093620] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8297.746576] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8299.532768] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8301.714686] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8301.812569] Lustre: lustre-OST0000: new disk, initializing [ 8301.814468] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8301.817161] Lustre: 286956:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8301.819755] Lustre: 286956:0:(osd_compat.c:1352:osd_object_spec_find()) Skipped 1 previous similar message [ 8301.847118] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8303.639639] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8307.254200] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8307.699995] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8307.703817] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8307.731429] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8314.614104] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8317.134235] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8325.586743] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8331.116218] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8384.481016] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8384.482296] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8384.483193] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8384.483544] LustreError: Skipped 1 previous similar message [ 8384.486788] Lustre: Skipped 1 previous similar message [ 8389.602278] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8389.604384] Lustre: Skipped 2 previous similar messages [ 8394.207120] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8394.257116] Lustre: server umount lustre-OST0000 complete [ 8399.840871] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8399.841597] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8399.846695] Lustre: Skipped 1 previous similar message [ 8401.623026] Lustre: server umount lustre-MDT0000 complete [ 8402.799104] LustreError: 284983:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194007 with bad export cookie 106483284959546565 [ 8402.802197] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8402.803178] LustreError: 284983:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8402.911516] Lustre: server umount lustre-MDT0001 complete [ 8405.666508] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8406.955392] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8408.533217] Lustre: server umount MGS complete [ 8412.195547] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8412.357300] LustreError: 290867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8412.363994] LustreError: 290880:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8412.390360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8413.706434] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8416.449560] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8417.870650] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8420.126120] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8421.812746] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8423.965332] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8425.888234] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8428.127493] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8462.304190] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8462.306544] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8462.311534] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8462.313343] Lustre: Skipped 1 previous similar message [ 8464.352409] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8466.442284] Lustre: server umount lustre-OST0000 complete [ 8467.936101] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8473.746861] Lustre: server umount lustre-MDT0000 complete [ 8474.591913] LustreError: 290868:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8474.597120] LustreError: 290868:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8474.853139] LustreError: 290848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194079 with bad export cookie 106483284959548406 [ 8474.855902] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8474.856707] LustreError: 290848:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8474.965433] Lustre: server umount lustre-MDT0001 complete [ 8478.212627] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8479.184550] Key type lgssc unregistered [ 8479.304390] LNet: 294847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8479.307690] LNetError: 294847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8479.319289] LNet: Removed LNI 192.168.204.106@tcp [ 8479.633803] Key type .llcrypt unregistered [ 8479.634927] Key type ._llcrypt unregistered [ 8490.746927] Key type ._llcrypt registered [ 8490.748161] Key type .llcrypt registered [ 8490.781648] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8496.617544] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8497.037000] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8497.043848] alg: No test for adler32 (adler32-zlib) [ 8497.882606] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 8497.965686] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8499.543119] Key type lgssc registered [ 8499.875481] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8510.857859] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8510.863853] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8511.948810] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8511.960175] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8511.997103] Lustre: lustre-MDT0000: new disk, initializing [ 8512.018245] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8512.024100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8513.288229] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8517.268710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8517.307686] Lustre: 298775:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8517.330524] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8517.332289] Lustre: Skipped 1 previous similar message [ 8517.364067] Lustre: lustre-MDT0001: new disk, initializing [ 8517.391729] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8517.402519] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8517.406248] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8518.607767] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8522.115248] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8523.796581] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8526.008806] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8526.110359] Lustre: lustre-OST0000: new disk, initializing [ 8526.112099] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8526.115485] Lustre: 300741:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8526.144185] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8526.653306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8526.656348] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8526.683675] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8528.006169] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8531.556625] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8534.883230] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8537.332578] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8546.929890] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8551.406446] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8588.258551] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8588.258955] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8588.263689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8588.269621] Lustre: Skipped 1 previous similar message [ 8593.377931] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8593.381015] Lustre: Skipped 1 previous similar message [ 8594.323929] Lustre: server umount lustre-OST0000 complete [ 8598.496803] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8598.497479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8598.501868] Lustre: Skipped 2 previous similar messages [ 8601.752958] Lustre: server umount lustre-MDT0000 complete [ 8602.938328] LustreError: 302393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194207 with bad export cookie 11922744253870179116 [ 8602.941716] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8602.941867] LustreError: 302393:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8603.054961] Lustre: server umount lustre-MDT0001 complete [ 8605.873082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8607.241653] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8608.778207] Lustre: server umount MGS complete [ 8612.343834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8612.504371] LustreError: 304710:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8612.526299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8613.912752] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8616.877038] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8616.994251] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8618.295480] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8620.569485] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8622.323786] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8624.440398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8626.432834] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8628.751293] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8658.400474] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8658.402848] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8658.403622] LustreError: Skipped 2 previous similar messages [ 8658.409733] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8658.412747] Lustre: Skipped 2 previous similar messages [ 8661.835343] Lustre: server umount lustre-OST0000 complete [ 8663.008647] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8663.011710] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8663.015772] Lustre: Skipped 1 previous similar message [ 8663.017305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8673.760329] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8673.762427] Lustre: Skipped 4 previous similar messages [ 8677.343253] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8677.416937] Lustre: server umount lustre-MDT0000 complete [ 8677.856691] LustreError: 304697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8677.861958] LustreError: 304697:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8678.662966] LustreError: 304677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194282 with bad export cookie 11922744253870180929 [ 8678.666062] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8678.666866] LustreError: 304677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8678.773401] Lustre: server umount lustre-MDT0001 complete [ 8682.281350] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 8683.258787] Key type lgssc unregistered [ 8683.392363] LNet: 308662:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8683.394747] LNetError: 308662:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8683.403403] LNet: Removed LNI 192.168.204.106@tcp [ 8683.722400] Key type .llcrypt unregistered [ 8683.723486] Key type ._llcrypt unregistered [ 8690.311736] Key type ._llcrypt registered [ 8690.313038] Key type .llcrypt registered [ 8690.348237] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8696.247930] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8696.574425] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8696.601611] alg: No test for adler32 (adler32-zlib) [ 8697.452119] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 8697.533284] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 8699.119141] Key type lgssc registered [ 8699.464251] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8706.912713] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8711.823749] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8711.828997] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8712.912963] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8712.923732] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8712.964513] Lustre: lustre-MDT0000: new disk, initializing [ 8712.987796] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8712.994129] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8714.274641] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8718.207619] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8718.302108] Lustre: lustre-OST0000: new disk, initializing [ 8718.303755] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8718.305466] Lustre: Skipped 1 previous similar message [ 8718.308321] Lustre: 312685:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8718.331715] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8719.808763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 8719.812328] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 8719.821371] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 8720.136568] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8924.640120] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8924.646331] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8925.325729] Lustre: server umount lustre-MDT0000 complete [ 8926.513353] LustreError: 312684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194530 with bad export cookie 13597992863497132584 [ 8926.515666] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8926.517182] LustreError: 312684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8932.622659] Lustre: server umount lustre-OST0000 complete [ 8940.686218] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 8943.790586] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 8955.164042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8955.245409] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8955.256669] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8955.292761] Lustre: lustre-MDT0000: new disk, initializing [ 8955.314978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8955.322533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8956.646102] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8960.719710] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8960.760316] Lustre: 316914:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8960.784532] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8960.786939] Lustre: Skipped 1 previous similar message [ 8960.824778] Lustre: lustre-MDT0001: new disk, initializing [ 8960.856295] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8960.868352] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8960.871838] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8962.176801] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8965.915503] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8967.839576] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8970.138491] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8970.234098] Lustre: lustre-OST0000: new disk, initializing [ 8970.235862] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8970.238640] Lustre: 318882:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8970.264558] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8972.207070] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 8975.811526] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8977.181500] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 8978.423587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8978.427816] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8978.454368] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8979.288153] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 2 sec [ 8980.611294] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 8980.684661] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 8983.521507] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8983.521950] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8983.522703] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8983.523867] LustreError: Skipped 1 previous similar message [ 8983.527187] Lustre: Skipped 1 previous similar message [ 8987.406421] Lustre: server umount lustre-OST0000 complete [ 8988.640353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8988.640952] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8988.644962] Lustre: Skipped 1 previous similar message [ 8988.647063] Lustre: Skipped 1 previous similar message [ 8993.760758] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8993.762359] Lustre: Skipped 1 previous similar message [ 8994.721546] Lustre: server umount lustre-MDT0000 complete [ 8995.926986] LustreError: 316906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194600 with bad export cookie 13597992863500073354 [ 8995.929130] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8995.930460] LustreError: 316906:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8996.045412] Lustre: server umount lustre-MDT0001 complete [ 8997.154727] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 8997.756709] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:10:01 (1786194601) [ 8999.946055] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9000.104206] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9001.380330] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9004.104453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9005.548125] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9007.733410] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9009.538308] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9011.654945] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9013.572415] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9016.080558] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9018.530837] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9018.597513] Lustre: lustre-OST0001: new disk, initializing [ 9018.599336] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9018.601910] Lustre: 324036:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9018.628350] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9018.630397] Lustre: Skipped 2 previous similar messages [ 9020.679813] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9026.044297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9026.047749] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9026.061153] LustreError: 324606:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9026.065520] LustreError: 324606:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9027.106548] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9027.109891] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9028.131622] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9028.135168] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 9029.672163] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9029.755764] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9030.178548] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 9030.182569] LustreError: 324612:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 9031.625142] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8c9e09e08800.ost_server_uuid 50 [ 9032.205978] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8c9e09e08800.ost_server_uuid in FULL state after 0 sec [ 9034.154478] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8c9e09e08800.ost_server_uuid 50 [ 9034.276137] LustreError: 324606:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 9034.279449] LustreError: 324606:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 9034.692397] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8c9e09e08800.ost_server_uuid in FULL state after 0 sec [ 9035.305986] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9046.497441] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9046.500113] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9046.501517] Lustre: Skipped 1 previous similar message [ 9052.415404] Lustre: server umount lustre-OST0001 complete [ 9054.687780] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9054.690505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9054.695630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9054.698086] Lustre: Skipped 4 previous similar messages [ 9055.712532] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9056.011957] Lustre: server umount lustre-OST0000 complete [ 9059.355721] Lustre: server umount lustre-MDT0000 complete [ 9060.494231] LustreError: 321186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194664 with bad export cookie 13597992863500074229 [ 9060.498147] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9060.498967] LustreError: 321186:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9060.605030] Lustre: server umount lustre-MDT0001 complete [ 9064.176115] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 9065.133913] Key type lgssc unregistered [ 9065.260419] LNet: 326721:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9065.263459] LNetError: 326721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9065.271293] LNet: Removed LNI 192.168.204.106@tcp [ 9065.561203] Key type .llcrypt unregistered [ 9065.562974] Key type ._llcrypt unregistered [ 9074.664375] Key type ._llcrypt registered [ 9074.665708] Key type .llcrypt registered [ 9074.703444] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9075.012577] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9075.040046] alg: No test for adler32 (adler32-zlib) [ 9075.887139] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 9075.976847] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 9077.559113] Key type lgssc registered [ 9077.877665] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9080.284300] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9080.290446] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9081.430454] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9082.719362] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9084.963891] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9087.172355] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9087.294904] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9088.525494] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9091.471850] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9091.582513] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9093.303812] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9095.629626] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9096.736319] LustreError: 330622:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9096.742924] LustreError: 330622:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9097.739203] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9097.801161] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9099.577083] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9101.828861] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9105.896713] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 9105.898037] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 9108.339346] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 9108.405265] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9184.735920] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9184.738445] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9184.743249] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9187.809269] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 9187.810670] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9187.815417] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9190.139757] Lustre: server umount lustre-OST0001 complete [ 9192.928495] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9192.930183] LustreError: 329797:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9192.931341] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9192.935831] LustreError: 329797:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9192.943899] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9192.946163] Lustre: Skipped 2 previous similar messages [ 9197.703803] Lustre: server umount lustre-OST0000 complete [ 9200.095813] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9200.098399] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9200.102139] Lustre: Skipped 1 previous similar message [ 9200.103868] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9205.089479] Lustre: server umount lustre-MDT0000 complete [ 9206.324118] LustreError: 328100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194810 with bad export cookie 14976862921944150362 [ 9206.328435] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9206.328730] LustreError: 328100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9206.459703] Lustre: server umount lustre-MDT0001 complete [ 9210.141954] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 9211.163783] Key type lgssc unregistered [ 9211.285639] LNet: 333053:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9211.288273] LNetError: 333053:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9211.301360] LNet: Removed LNI 192.168.204.106@tcp [ 9211.619149] Key type .llcrypt unregistered [ 9211.620340] Key type ._llcrypt unregistered [ 9229.285125] Key type ._llcrypt registered [ 9229.286537] Key type .llcrypt registered [ 9229.324210] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9229.632596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9229.670173] alg: No test for adler32 (adler32-zlib) [ 9230.524691] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 9230.608631] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 9232.191203] Key type lgssc registered [ 9232.543632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9235.492489] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9235.497447] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9236.634971] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9237.980741] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9240.650255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9240.775868] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9242.059868] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9243.016213] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 9245.119447] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9245.239352] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9246.243546] LustreError: 336805:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9246.246464] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 9246.250526] LustreError: 336805:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9247.051795] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9249.794292] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9249.885612] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 9250.929991] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 9251.876589] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9256.040125] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9258.717760] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 9266.660150] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9266.661383] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9266.661554] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9266.664910] Lustre: Skipped 3 previous similar messages [ 9266.670178] Lustre: Skipped 3 previous similar messages [ 9268.509658] Lustre: server umount lustre-MDT0000 complete [ 9269.738351] LustreError: 336456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194874 with bad export cookie 9973472410612474277 [ 9269.741882] LustreError: 336456:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9269.742390] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9269.852437] Lustre: server umount lustre-MDT0001 complete [ 9280.963175] Lustre: server umount lustre-OST0000 complete [ 9292.448651] Lustre: server umount lustre-OST0001 complete [ 9298.076775] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9301.524608] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9301.658973] LustreError: 340578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9301.676036] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9302.995987] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9305.790271] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9307.284931] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9308.279959] Lustre: 341717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 9310.429370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9310.558523] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9310.561368] Lustre: Skipped 1 previous similar message [ 9311.586118] LustreError: 342070:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9311.590671] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 9312.478214] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9315.318744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9316.457319] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 9317.326425] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9320.385270] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 9321.520591] Lustre: 343587:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 9326.559943] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9326.562122] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9326.567963] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9328.534828] Lustre: server umount lustre-MDT0000 complete [ 9329.710553] LustreError: 340558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786194934 with bad export cookie 9973472410612475964 [ 9329.714755] LustreError: 340558:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9329.716140] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9329.847438] Lustre: server umount lustre-MDT0001 complete [ 9341.381814] Lustre: server umount lustre-OST0000 complete [ 9351.765377] Lustre: server umount lustre-OST0001 complete [ 9352.794178] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 9353.308159] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 9353.880637] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:15:57 (1786194957) [ 9361.011475] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9364.088177] loop0: detected capacity change from 0 to 1099511627776 [ 9364.247980] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 9364.251742] *** VALIDATE xfs *** [ 9364.256927] XFS (loop0): Mounting V5 Filesystem [ 9364.265225] XFS (loop0): Ending clean mount [ 9364.645864] loop1: detected capacity change from 0 to 18691697672192 [ 9383.811413] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [ 9387.132292] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9398.974419] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9399.062090] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9399.074517] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9399.119561] Lustre: lustre-MDT0000: new disk, initializing [ 9399.142328] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9399.144492] Lustre: Skipped 1 previous similar message [ 9399.149339] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9400.539729] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9404.701056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9404.750043] Lustre: 348694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9404.774629] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9404.777175] Lustre: Skipped 1 previous similar message [ 9404.815852] Lustre: lustre-MDT0001: new disk, initializing [ 9404.854213] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9404.858710] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9406.081317] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9409.612616] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9411.293316] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9413.413348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9413.529667] Lustre: lustre-OST0000: new disk, initializing [ 9413.531408] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9413.534170] Lustre: 350662:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9415.288386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9415.291583] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9415.325564] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9415.393432] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9418.933686] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9420.262736] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 9420.336167] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 9421.569391] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 9421.635153] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 9425.889751] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9425.889931] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9425.891326] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9425.891330] Lustre: Skipped 4 previous similar messages [ 9425.893459] Lustre: Skipped 4 previous similar messages [ 9425.896421] LustreError: Skipped 1 previous similar message [ 9428.371969] Lustre: server umount lustre-OST0000 complete [ 9430.495892] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9430.498266] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9430.502365] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9430.504107] Lustre: Skipped 1 previous similar message [ 9435.681355] Lustre: server umount lustre-MDT0000 complete [ 9436.127963] LustreError: 348707:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9436.132186] LustreError: 348707:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9436.946162] LustreError: 348688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195041 with bad export cookie 9973472410612477497 [ 9436.949480] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9436.949608] LustreError: 348688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9437.059404] Lustre: server umount lustre-MDT0001 complete [ 9439.604614] XFS (loop0): Unmounting Filesystem [ 9441.052051] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 09:17:24 (1786195044) [ 9443.162978] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9443.288376] LustreError: 353215:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9443.308860] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9443.311278] Lustre: Skipped 2 previous similar messages [ 9444.568427] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9447.198368] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9448.713302] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9451.130713] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9453.019659] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9455.249910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9457.362336] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9459.815616] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9478.113399] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9478.115542] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9478.117458] LustreError: Skipped 1 previous similar message [ 9478.118930] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9478.118940] Lustre: Skipped 2 previous similar messages [ 9478.121073] Lustre: Skipped 2 previous similar messages [ 9479.765851] Lustre: server umount lustre-OST0000 complete [ 9487.200559] Lustre: server umount lustre-MDT0000 complete [ 9488.352183] LustreError: 353216:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9488.356593] LustreError: 353216:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9488.483607] LustreError: 353196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786195092 with bad export cookie 9973472410612478379 [ 9488.487217] LustreError: 353196:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9488.488406] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9488.618141] Lustre: server umount lustre-MDT0001 complete [ 9492.285732] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [ 9493.359975] Key type lgssc unregistered [ 9493.492564] LNet: 356864:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9493.496265] LNetError: 356864:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9493.507370] LNet: Removed LNI 192.168.204.106@tcp [ 9493.851108] Key type .llcrypt unregistered [ 9493.852159] Key type ._llcrypt unregistered [ 9504.219110] Key type ._llcrypt registered [ 9504.220774] Key type .llcrypt registered [ 9504.259238] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [ 9504.696043] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9504.703045] alg: No test for adler32 (adler32-zlib) [ 9505.547784] Lustre: Lustre: Build Version: 2.17.56_50_g336d872 [ 9505.632078] LNet: Added LNI 192.168.204.106@tcp [8/256/0/180] [ 9507.207164] Key type lgssc registered [ 9507.555783] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9510.092417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9510.100873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9511.251052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9512.598576] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9515.348674] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9515.484419] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9516.871082] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9519.282414] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9521.153857] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9523.372427] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9523.496214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9525.429968] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9527.860317] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9543.768706] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [ 9546.213594] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9546.213766] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9546.219047] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9546.222174] Lustre: Skipped 1 previous similar message [ 9547.350635] Lustre: server umount lustre-MDT0001 complete [ 9548.881511] LustreError: 360893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9548.886628] LustreError: 360893:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9550.867647] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9550.903363] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9550.906107] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9550.919384] Lustre: 358394:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9551.038143] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9552.406215] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9556.454656] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9556.965018] LustreError: 362136:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9556.968867] LustreError: 362136:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9556.972747] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9556.992230] LustreError: 362139:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [ 9556.995110] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 9556.995110] [ 9856.085640] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [ 9858.529427] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9858.529823] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 9858.531960] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9858.533890] Lustre: Skipped 2 previous similar messages [ 9863.649147] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9863.650843] Lustre: Skipped 3 previous similar messages [ 9869.279079] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9869.343596] Lustre: server umount lustre-MDT0001 complete [ 9871.440959] LustreError: 362291:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9872.787884] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9872.824103] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 9872.827064] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9872.844080] Lustre: 358395:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9872.958821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9874.310227] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [ 9877.990048] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 9880.550722] LustreError: 374815:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 9880.553498] LustreError: 374815:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 9880.556485] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 9880.560102] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 9880.577926] LustreError: 374823:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 9880.580444] LustreError: 374823:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 9880.582628] LustreError: 374823:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -17 [ 9880.585229] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.106@tcp [ 9880.585229] [10178.647099] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.6@tcp (stopping) [10178.650329] Lustre: Skipped 3 previous similar messages [10180.066461] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [10180.066560] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10180.075915] Lustre: Skipped 1 previous similar message [10183.446199] Lustre: server umount lustre-MDT0001 complete [10183.762415] LustreError: 358401:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10186.852141] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10186.885173] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [10186.886791] Lustre: Found index 1 for lustre-MDT0001, updating log [10186.899046] Lustre: 358394:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10187.004500] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [10188.422335] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10192.362435] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [10196.965472] LustreError: 387587:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [10196.968373] LustreError: 387587:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [10196.971136] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [10196.972900] LustreError: 358398:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [10196.991801] LustreError: 387590:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [10196.996563] LustreError: 387590:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.106@tcp: cfg command failed: rc = -114 [10196.999235] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [10196.999235] [10504.675023] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10504.675186] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10504.675622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10504.675626] Lustre: Skipped 2 previous similar messages [10504.684814] Lustre: Skipped 1 previous similar message [10508.441895] Lustre: server umount lustre-MDT0000 complete [10509.607178] LustreError: 386912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196113 with bad export cookie 1208458198558691092 [10509.611339] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10509.612727] LustreError: 386912:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10509.791930] LustreError: 360893:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10509.792147] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10509.799452] LustreError: 360893:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10509.802639] Lustre: Skipped 1 previous similar message [10514.911940] LustreError: 358402:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10515.752242] Lustre: server umount lustre-MDT0001 complete [10516.985564] Lustre: server umount lustre-OST0000 complete [10519.821347] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [10523.043801] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [10534.343646] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 09:35:37 (1786196137) [10536.512621] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10536.591791] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10536.600736] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10536.634639] Lustre: lustre-MDT0000: new disk, initializing [10536.654937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10536.661965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10537.960428] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10541.964226] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10542.020397] Lustre: 402097:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10542.052032] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10542.054227] Lustre: Skipped 1 previous similar message [10542.091420] Lustre: lustre-MDT0001: new disk, initializing [10542.129807] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10542.147086] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10542.150366] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10543.569978] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10547.110970] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10548.810144] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10551.061868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10551.174542] Lustre: lustre-OST0000: new disk, initializing [10551.176294] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10551.178808] Lustre: 404062:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10551.207279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10553.053367] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_default_debug -1 all [10556.645332] Lustre: DEBUG MARKER: oleg406-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10561.018023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10561.021820] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10561.051223] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10566.114301] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10566.115529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10566.117652] Lustre: Skipped 2 previous similar messages [10566.121208] Lustre: Skipped 6 previous similar messages [10570.786149] Lustre: server umount lustre-MDT0000 complete [10571.232955] LustreError: 402106:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10571.237771] LustreError: 402106:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10572.000299] LustreError: 402087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786196176 with bad export cookie 1208458198558693031 [10572.003215] LustreError: 402904:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [10572.003260] LustreError: MGC192.168.204.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10572.004230] LustreError: 402087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10572.115873] Lustre: server umount lustre-MDT0001 complete [10583.575145] Lustre: server umount lustre-OST0000 complete [10593.001335] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing set_hostid [10596.250154] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing load_modules_local [10606.598672] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 10240 sec ========= 09:36:50 (1786196210) [10607.181774] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 09:36:50 (1786196210) === [10608.366635] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 09:36:51 (1786196211) === [10621.818163] Lustre: DEBUG MARKER: oleg406-server.virtnet: executing unload_modules_local [10622.934984] Key type lgssc unregistered [10623.073246] LNet: 410421:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10623.075738] LNetError: 410421:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10623.087295] LNet: Removed LNI 192.168.204.106@tcp [10623.381265] Key type .llcrypt unregistered [10623.383045] Key type ._llcrypt unregistered