[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 463873625 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001014] APIC: Switch to symmetric I/O mode setup [ 0.003348] x2apic enabled [ 0.004012] Switched APIC routing to physical x2apic. [ 0.005023] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008048] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009020] pid_max: default: 32768 minimum: 301 [ 0.011136] LSM: Security Framework initializing [ 0.013033] Yama: becoming mindful. [ 0.014046] SELinux: Initializing. [ 0.015106] *** VALIDATE selinux *** [ 0.023655] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028690] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030051] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032045] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.033149] *** VALIDATE tmpfs *** [ 0.035122] *** VALIDATE proc *** [ 0.036221] *** VALIDATE cgroup *** [ 0.037017] *** VALIDATE cgroup2 *** [ 0.039240] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.040201] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.041019] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.042041] Spectre V2 : User space: Vulnerable [ 0.043014] Speculative Store Bypass: Vulnerable [ 0.046927] debug: unmapping init [mem 0xffffffffb9859000-0xffffffffb9860fff] [ 0.049000] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.049808] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.050034] ... version: 2 [ 0.051017] ... bit width: 48 [ 0.052013] ... generic registers: 4 [ 0.053000] ... value mask: 0000ffffffffffff [ 0.054032] ... max period: 00007fffffffffff [ 0.055018] ... fixed-purpose events: 3 [ 0.056020] ... event mask: 000000070000000f [ 0.058305] rcu: Hierarchical SRCU implementation. [ 0.060578] smp: Bringing up secondary CPUs ... [ 0.061515] x86: Booting SMP configuration: [ 0.062037] .... node #0, CPUs: #1 #2 #3 [ 0.065786] smp: Brought up 1 node, 4 CPUs [ 0.067015] smpboot: Max logical packages: 1 [ 0.068023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.232363] node 0 deferred pages initialised in 162ms [ 0.236008] devtmpfs: initialized [ 0.237000] x86/mm: Memory block size: 128MB [ 0.239548] gcov: version magic: 0x41383552 [ 0.243288] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.244135] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.245165] pinctrl core: initialized pinctrl subsystem [ 0.247146] [ 0.247727] ************************************************************* [ 0.250015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.252014] ** ** [ 0.254016] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.257017] ** ** [ 0.259013] ** This means that this kernel is built to expose internal ** [ 0.261017] ** IOMMU data structures, which may compromise security on ** [ 0.264016] ** your system. ** [ 0.266014] ** ** [ 0.268015] ** If you see this message and you are not debugging the ** [ 0.270014] ** kernel, report this immediately to your vendor! ** [ 0.272016] ** ** [ 0.274015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.276014] ************************************************************* [ 0.278657] NET: Registered protocol family 16 [ 0.279372] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.282072] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.284072] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.287544] cpuidle: using governor menu [ 0.288649] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.290540] PCI: Using configuration type 1 for base access [ 0.293164] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.302140] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.305051] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.310040] cryptd: max_cpu_qlen set to 1000 [ 0.313361] ACPI: Added _OSI(Module Device) [ 0.314019] ACPI: Added _OSI(Processor Device) [ 0.316026] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.317016] ACPI: Added _OSI(Processor Aggregator Device) [ 0.322890] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.329331] ACPI: Interpreter enabled [ 0.330064] ACPI: PM: (supports S0 S3 S4 S5) [ 0.332015] ACPI: Using IOAPIC for interrupt routing [ 0.333101] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.336425] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.345985] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.348047] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.350018] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.352101] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.357430] acpiphp: Slot [2] registered [ 0.358125] acpiphp: Slot [5] registered [ 0.360394] acpiphp: Slot [6] registered [ 0.361136] acpiphp: Slot [7] registered [ 0.363136] acpiphp: Slot [8] registered [ 0.364119] acpiphp: Slot [9] registered [ 0.365095] acpiphp: Slot [10] registered [ 0.366065] acpiphp: Slot [3] registered [ 0.367014] acpiphp: Slot [4] registered [ 0.368115] acpiphp: Slot [11] registered [ 0.369117] acpiphp: Slot [12] registered [ 0.371107] acpiphp: Slot [13] registered [ 0.372106] acpiphp: Slot [14] registered [ 0.374100] acpiphp: Slot [15] registered [ 0.375122] acpiphp: Slot [16] registered [ 0.376120] acpiphp: Slot [17] registered [ 0.378131] acpiphp: Slot [18] registered [ 0.379113] acpiphp: Slot [19] registered [ 0.380105] acpiphp: Slot [20] registered [ 0.382150] acpiphp: Slot [21] registered [ 0.383104] acpiphp: Slot [22] registered [ 0.385106] acpiphp: Slot [23] registered [ 0.386145] acpiphp: Slot [24] registered [ 0.387104] acpiphp: Slot [25] registered [ 0.388110] acpiphp: Slot [26] registered [ 0.390174] acpiphp: Slot [27] registered [ 0.391099] acpiphp: Slot [28] registered [ 0.392068] acpiphp: Slot [29] registered [ 0.393110] acpiphp: Slot [30] registered [ 0.394089] acpiphp: Slot [31] registered [ 0.396094] PCI host bridge to bus 0000:00 [ 0.397022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.399031] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.401030] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.404029] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.406027] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.409032] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.411185] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.414011] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.417319] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.424545] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.428053] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.431022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.433020] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.435017] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.437578] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.440361] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.443047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.445760] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.450034] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.463022] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.467020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.473110] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.481023] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.490020] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.509019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.518152] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.526019] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.532030] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.546034] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.556380] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.563021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.572020] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.586022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.594477] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.601022] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.606022] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.616016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.624829] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.629014] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.633016] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.645016] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.652714] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.657014] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.662016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.672016] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.681310] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.684373] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.686346] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.687225] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.689160] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.692120] iommu: Default domain type: Passthrough [ 0.694385] SCSI subsystem initialized [ 0.696189] ACPI: bus type USB registered [ 0.697094] usbcore: registered new interface driver usbfs [ 0.699109] usbcore: registered new interface driver hub [ 0.700070] usbcore: registered new device driver usb [ 0.701131] pps_core: LinuxPPS API ver. 1 registered [ 0.703016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.705070] PTP clock support registered [ 0.707118] EDAC MC: Ver: 3.0.0 [ 0.708750] PCI: Using ACPI for IRQ routing [ 0.709900] NetLabel: Initializing [ 0.710013] NetLabel: domain hash size = 128 [ 0.711012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.712243] NetLabel: unlabeled traffic allowed by default [ 0.714530] vgaarb: loaded [ 0.716222] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.718010] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.725275] clocksource: Switched to clocksource kvm-clock [ 0.833445] VFS: Disk quotas dquot_6.6.0 [ 0.834928] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.837346] *** VALIDATE ramfs *** [ 0.838706] *** VALIDATE hugetlbfs *** [ 0.840342] pnp: PnP ACPI init [ 0.842818] pnp: PnP ACPI: found 6 devices [ 0.858902] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.861735] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.864469] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.866611] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.869324] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.872626] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.875575] NET: Registered protocol family 2 [ 0.878454] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.883833] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.887579] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.893078] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.897031] TCP: Hash tables configured (established 65536 bind 65536) [ 0.899501] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.902570] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.905427] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.908636] NET: Registered protocol family 1 [ 0.911341] RPC: Registered named UNIX socket transport module. [ 0.912723] RPC: Registered udp transport module. [ 0.913772] RPC: Registered tcp transport module. [ 0.916518] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.918852] NET: Registered protocol family 44 [ 0.920574] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.922461] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.924099] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.926922] PCI: CLS 0 bytes, default 64 [ 0.928953] Unpacking initramfs... [ 2.375201] debug: unmapping init [mem 0xffff9258fcc54000-0xffff9258fffbffff] [ 2.378971] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.380973] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.383088] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.888664] Initialise system trusted keyrings [ 2.889761] Key type blacklist registered [ 2.891251] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.899205] zbud: loaded [ 2.902061] *** VALIDATE nfs *** [ 2.903346] *** VALIDATE nfs4 *** [ 2.906265] pstore: using deflate compression [ 2.909729] Platform Keyring initialized [ 3.020461] NET: Registered protocol family 38 [ 3.022616] Key type asymmetric registered [ 3.024590] Asymmetric key parser 'x509' registered [ 3.026702] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.030229] io scheduler mq-deadline registered [ 3.032100] io scheduler kyber registered [ 3.034123] io scheduler bfq registered [ 3.036209] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.039470] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.042625] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.045681] ACPI: Power Button [PWRF] [ 3.051421] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.068432] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.082071] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.088725] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.108954] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.139087] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.173455] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.181995] Non-volatile memory driver v1.3 [ 3.183822] Linux agpgart interface v0.103 [ 3.221897] virtio_blk virtio1: [vda] 149392 512-byte logical blocks (76.5 MB/72.9 MiB) [ 3.226468] vda: detected capacity change from 0 to 76488704 [ 3.242532] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.245479] vdb: detected capacity change from 0 to 1073741824 [ 3.260036] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.262341] vdc: detected capacity change from 0 to 2621440000 [ 3.280706] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.283293] vdd: detected capacity change from 0 to 2621440000 [ 3.296291] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.299227] vde: detected capacity change from 0 to 4294967296 [ 3.312798] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.315751] vdf: detected capacity change from 0 to 4294967296 [ 3.322388] libphy: Fixed MDIO Bus: probed [ 3.342530] usbcore: registered new interface driver usbserial_generic [ 3.345543] usbserial: USB Serial support registered for generic [ 3.348140] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.353287] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.355687] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.360686] mousedev: PS/2 mouse device common for all mice [ 3.363710] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.365467] rtc_cmos 00:05: RTC can wake from S4 [ 3.377112] rtc_cmos 00:05: registered as rtc0 [ 3.378821] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.382981] intel_pstate: CPU model not supported [ 3.386074] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.390929] hid: raw HID events driver (C) Jiri Kosina [ 3.394190] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.396202] usbcore: registered new interface driver usbhid [ 3.402455] usbhid: USB HID core driver [ 3.404149] drop_monitor: Initializing network drop monitor service [ 3.406728] Initializing XFRM netlink socket [ 3.408927] NET: Registered protocol family 10 [ 3.412221] Segment Routing with IPv6 [ 3.413733] NET: Registered protocol family 17 [ 3.416290] mpls_gso: MPLS GSO support [ 3.423232] RAS: Correctable Errors collector initialized. [ 3.425596] AVX version of gcm_enc/dec engaged. [ 3.427351] AES CTR mode by8 optimization enabled [ 3.492768] sched_clock: Marking stable (3492689461, 0)->(4426875734, -934186273) [ 3.497378] registered taskstats version 1 [ 3.499475] Loading compiled-in X.509 certificates [ 3.502075] zswap: loaded using pool lzo/zbud [ 3.537466] Key type big_key registered [ 3.556235] Key type encrypted registered [ 3.560507] ima: No TPM chip found, activating TPM-bypass! [ 3.562758] ima: Allocated hash algorithm: sha1 [ 3.564882] ima: No architecture policies found [ 3.567179] evm: Initialising EVM extended attributes: [ 3.569419] evm: security.selinux [ 3.570800] evm: security.ima [ 3.572234] evm: security.capability [ 3.573661] evm: HMAC attrs: 0x1 [ 3.576344] rtc_cmos 00:05: setting system clock to 2026-08-10 22:38:19 UTC (1786401499) [ 3.583123] debug: unmapping init [mem 0xffffffffba803000-0xffffffffba9fffff] [ 3.586531] debug: unmapping init [mem 0xffffffffb9582000-0xffffffffb9858fff] [ 3.595091] Write protecting the kernel read-only data: 28672k [ 3.598295] debug: unmapping init [mem 0xffffffffb7c03000-0xffffffffb7dfffff] [ 3.601064] debug: unmapping init [mem 0xffffffffb8514000-0xffffffffb85fffff] [ 3.638764] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.648701] systemd[1]: Detected virtualization kvm. [ 3.651254] systemd[1]: Detected architecture x86-64. [ 3.653604] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.685788] systemd[1]: No hostname configured. [ 3.688013] systemd[1]: Set hostname to . [ 3.690459] random: systemd: uninitialized urandom read (16 bytes read) [ 3.693253] systemd[1]: Initializing machine ID from random generator. [ 3.768681] random: ln: uninitialized urandom read (6 bytes read) [ 3.867090] random: systemd: uninitialized urandom read (16 bytes read) [ 3.869513] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 3.873491] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 3.876420] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Swap. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Sockets. Starting Setup Virtual Console... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.548053] device-mapper: uevent: version 1.0.3 [ 4.550444] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.288361] random: fast init done [ 5.296169] virtio_net virtio0 ens2: renamed from eth0 [ 5.375432] scsi host0: ata_piix [ 5.388329] scsi host1: ata_piix [ 5.389784] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.391935] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.822473] dracut-initqueue[593]: RTNETLINK answers: File exists [ 10.080106] random: crng init done [ 10.081622] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.607597] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.865984] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.141503] SELinux: Disabled at runtime. [ 12.206554] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.214354] systemd[1]: Detected virtualization kvm. [ 12.216212] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.767807] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.771492] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.776310] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.782542] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.785996] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.794233] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.798815] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Listening on Process Core Dump Socket. Mounting Kernel Debug File System... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ 12.904470] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Mounting Huge Pages File System... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ 13.362415] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /mnt. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ 13.806621] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.896725] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.968538] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.994796] EDAC sbridge: Ver: 1.1.2 [ 15.715424] Key type dns_resolver registered [ 16.047089] NFS: Registering the id_resolver key type [ 16.049726] Key type id_resolver registered [ 16.051645] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Authorization Manager. [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg249-server login: [ 61.784513] hrtimer: interrupt took 2515074 ns [ 65.630490] libcfs: loading out-of-tree module taints kernel. [ 65.677474] Key type ._llcrypt registered [ 65.680775] Key type .llcrypt registered [ 65.805454] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 85.355615] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 87.111972] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 87.138716] alg: No test for adler32 (adler32-zlib) [ 88.425590] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 89.203204] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 90.879199] Key type lgssc registered [ 92.601601] Lustre: Echo OBD driver; http://www.lustre.org/ [ 109.787724] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 153.047180] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 167.729090] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 167.777498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 169.042205] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 169.066452] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 169.151681] Lustre: lustre-MDT0000: new disk, initializing [ 169.253279] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 169.268991] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 173.728782] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 186.131648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 186.241606] Lustre: 6507:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 186.269709] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 186.273778] Lustre: Skipped 1 previous similar message [ 186.342530] Lustre: lustre-MDT0001: new disk, initializing [ 186.403329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 186.433227] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 186.448962] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 190.337882] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 194.793765] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 204.933647] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 205.157217] Lustre: lustre-OST0000: new disk, initializing [ 205.160576] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 205.164504] Lustre: 8442:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 205.209899] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 212.012666] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 212.013935] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 212.048381] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 212.107587] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 225.629574] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.774143] Lustre: lustre-OST0001: new disk, initializing [ 225.777609] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.782885] Lustre: 9515:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.862434] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 232.502577] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 232.517827] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 232.576911] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 232.596277] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 247.785843] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 257.898199] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 266.590611] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing check_logdir /tmp/testlogs/ [ 272.764896] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing yml_node [ 276.824723] Lustre: DEBUG MARKER: Client: 2.17.54.108 [ 279.294453] Lustre: DEBUG MARKER: MDS: 2.17.54.108 [ 282.300581] Lustre: DEBUG MARKER: OSS: 2.17.54.108 [ 284.226643] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Mon Aug 10 18:42:57 EDT 2026 [ 303.613467] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 305.294148] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 324.578640] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 324.582339] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.620290] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 325.088833] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 325.105767] Lustre: Skipped 2 previous similar messages [ 330.207920] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 330.224810] Lustre: Skipped 7 previous similar messages [ 335.327572] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 335.329615] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 335.348301] Lustre: Skipped 3 previous similar messages [ 335.495672] Lustre: server umount lustre-MDT0000 complete [ 338.860070] LustreError: 10333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786401834 with bad export cookie 15363046396896050930 [ 338.863508] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 338.872897] LustreError: 10333:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 340.449346] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 340.453413] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 340.473301] Lustre: Skipped 1 previous similar message [ 340.482124] Lustre: Skipped 1 previous similar message [ 348.576270] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 348.590220] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 348.596844] LustreError: 6513:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 348.605738] Lustre: Skipped 3 previous similar messages [ 353.248322] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 353.466969] Lustre: server umount lustre-MDT0001 complete [ 357.806780] Lustre: server umount lustre-OST0000 complete [ 362.218865] Lustre: server umount lustre-OST0001 complete [ 369.490830] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 379.622876] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 422.175309] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 422.362206] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 422.382847] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 422.469592] Lustre: lustre-MDT0000: new disk, initializing [ 422.545311] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 422.568382] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 428.482423] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 440.881820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 441.017482] Lustre: 15072:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 441.109299] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 441.113473] Lustre: Skipped 1 previous similar message [ 441.215489] Lustre: lustre-MDT0001: new disk, initializing [ 441.392455] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 441.465228] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 441.478380] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 446.474710] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 457.070302] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 463.650224] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 471.897326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 472.236809] Lustre: lustre-OST0000: new disk, initializing [ 472.239676] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 472.244843] Lustre: 17043:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 472.416676] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 473.540958] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 473.544363] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 473.614535] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 480.633418] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 491.883187] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 497.263870] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 497.477578] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 503.070304] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 503.261567] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 509.419798] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 509.465334] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 509.465571] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 509.467526] Lustre: Skipped 1 previous similar message [ 509.486347] LustreError: Skipped 1 previous similar message [ 511.551365] Lustre: server umount lustre-OST0000 complete [ 518.116039] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 518.120063] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 521.569617] Lustre: server umount lustre-MDT0000 complete [ 524.768767] LustreError: 16206:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.788816] LustreError: 16206:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 524.888148] LustreError: 15065:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402020 with bad export cookie 15363046396896052456 [ 524.898677] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 525.236493] Lustre: server umount lustre-MDT0001 complete [ 531.295413] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 18:47:04 (1786402024) [ 538.532598] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 538.998197] LustreError: 19309:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 539.107560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 543.679591] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 544.226254] LustreError: 19310:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 544.242401] LustreError: 19310:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 548.320695] LustreError: 19309:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 553.122128] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 553.454841] Lustre: lustre-MDT0001: Not available for connect from 0@lo (not set up) [ 553.466910] Lustre: Skipped 4 previous similar messages [ 559.418826] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 567.449129] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 572.934127] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 580.095910] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 588.559472] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 597.354745] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 604.502955] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 614.373711] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 614.379314] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 614.396425] Lustre: Skipped 3 previous similar messages [ 614.408183] Lustre: Skipped 2 previous similar messages [ 617.319728] Lustre: server umount lustre-MDT0000 complete [ 619.489027] LustreError: 20253:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 619.511253] LustreError: 20253:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 621.072802] LustreError: 19292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402116 with bad export cookie 15363046396896053345 [ 621.082991] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 621.091775] LustreError: 19292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 621.514031] Lustre: server umount lustre-MDT0001 complete [ 635.822366] Lustre: server umount lustre-OST0000 complete [ 659.221212] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 669.329948] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 669.760282] LustreError: 23694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 669.898844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 669.905367] Lustre: Skipped 2 previous similar messages [ 675.424928] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 685.279591] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 690.519583] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 693.611220] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 702.213079] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 709.586568] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 721.154426] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 721.250064] Lustre: lustre-OST0001: new disk, initializing [ 721.251975] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 721.255440] Lustre: 26009:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 722.658986] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 722.674487] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 722.949525] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 729.094970] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 738.817586] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 743.065676] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 757.735793] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 757.743458] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 757.769366] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 762.882617] Lustre: server umount lustre-MDT0000 complete [ 763.894593] LustreError: 23694:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 763.922826] LustreError: 23694:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 766.327204] LustreError: 23675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402262 with bad export cookie 15363046396896054192 [ 766.330747] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 766.342761] LustreError: 23675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 766.725759] Lustre: server umount lustre-MDT0001 complete [ 780.833126] Lustre: server umount lustre-OST0000 complete [ 795.033756] Lustre: server umount lustre-OST0001 complete [ 811.685953] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 821.578887] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 822.078560] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 822.088050] Lustre: Skipped 3 previous similar messages [ 826.608708] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 831.458444] LustreError: 29033:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 831.466602] LustreError: 29033:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 834.917488] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 839.462348] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 842.400642] Lustre: 30180:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 849.534268] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 856.650936] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 866.818473] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 875.543118] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 884.182497] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 889.347291] Lustre: 32060:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 902.119186] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 902.133465] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 902.158862] Lustre: Skipped 3 previous similar messages [ 902.172646] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 902.178150] Lustre: Skipped 4 previous similar messages [ 904.798704] Lustre: server umount lustre-MDT0000 complete [ 910.640911] LustreError: 29015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402406 with bad export cookie 15363046396896055487 [ 910.645842] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 910.652599] LustreError: 29015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 911.019045] Lustre: server umount lustre-MDT0001 complete [ 926.614811] Lustre: server umount lustre-OST0000 complete [ 929.378375] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786402409/real 1786402409] req@ffff92597c699c00 x1873177828701568/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786402425 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 931.562235] Lustre: server umount lustre-OST0001 complete [ 941.792988] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:53:54 (1786402434) [ 972.367416] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 982.721876] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 983.070582] LustreError: 34867:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 983.105458] LustreError: 34867:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [ 987.656352] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 996.081512] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1001.073354] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1004.189118] Lustre: 36014:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1010.297068] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1017.265649] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1025.483936] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1032.347195] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1041.711588] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1046.092565] Lustre: 37894:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1054.696385] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1054.718694] Lustre: Skipped 6 previous similar messages [ 1059.599628] Lustre: server umount lustre-MDT0000 complete [ 1063.333501] LustreError: 37896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402559 with bad export cookie 15363046396896056593 [ 1063.335161] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1063.343220] LustreError: 37896:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1063.742328] Lustre: server umount lustre-MDT0001 complete [ 1078.169214] Lustre: server umount lustre-OST0000 complete [ 1080.287877] Lustre: 3642:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786402560/real 1786402560] req@ffff925974522a00 x1873177828750848/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786402576 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1081.935574] Lustre: server umount lustre-OST0001 complete [ 1091.973955] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:56:24 (1786402584) [ 1109.159990] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1122.067615] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1122.524628] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1122.534720] Lustre: Skipped 7 previous similar messages [ 1127.686944] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1137.145293] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1142.901599] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1145.916656] Lustre: 41183:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1153.425469] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1161.115649] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1170.215095] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1177.204654] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1185.739887] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1189.737503] Lustre: 43061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1190.822313] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1190.828264] Lustre: Skipped 1 previous similar message [ 1195.280627] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.49@tcp (stopping) [ 1195.286933] Lustre: Skipped 8 previous similar messages [ 1196.525457] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1196.548228] Lustre: Skipped 4 previous similar messages [ 1200.838690] Lustre: server umount lustre-MDT0000 complete [ 1205.826083] LustreError: 40014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402701 with bad export cookie 15363046396896057699 [ 1205.838447] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1205.852192] LustreError: 40014:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1212.469704] Lustre: server umount lustre-MDT0001 complete [ 1224.160238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1230.308588] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xd534814d12d59da7 [ 1230.321201] Lustre: MGC192.168.202.149@tcp: Connection restored to 0@lo (at 0@lo) [ 1236.797557] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1240.230972] LustreError: 43937:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1240.249761] LustreError: 43937:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 32 previous similar messages [ 1240.252386] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1246.100473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1251.831804] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1251.842872] Lustre: Skipped 1 previous similar message [ 1252.288558] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1261.310405] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1263.162258] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1270.024437] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 1272.179641] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1282.557745] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1284.991985] Lustre: server umount lustre-MDT0000 complete [ 1288.580460] LustreError: 40015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402784 with bad export cookie 15363046396896058791 [ 1288.585518] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1288.595107] LustreError: 40015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1289.083257] Lustre: server umount lustre-MDT0001 complete [ 1303.778480] Lustre: server umount lustre-OST0000 complete [ 1308.137713] Lustre: 3641:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786402788/real 1786402788] req@ffff9258431cb100 x1873177828840832/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786402804 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1308.788500] Lustre: server umount lustre-OST0001 complete [ 1320.569380] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 19:00:13 (1786402813) [ 1343.040853] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1356.222399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1362.244875] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1374.229894] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1380.004449] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1383.496257] Lustre: 48840:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1392.148200] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1401.136554] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1413.211588] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1422.135554] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1430.708838] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1435.802607] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1451.703173] Lustre: server umount lustre-MDT0000 complete [ 1455.754864] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786402951 with bad export cookie 15363046396896059610 [ 1455.774209] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1455.779505] LustreError: 47655:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 1456.160160] Lustre: server umount lustre-MDT0001 complete [ 1470.755726] Lustre: server umount lustre-OST0000 complete [ 1486.043070] Lustre: server umount lustre-OST0001 complete [ 1497.003847] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 19:03:10 (1786402990) [ 1499.938789] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1501.904510] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 19:03:15 (1786402995) [ 1514.425958] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 1525.669828] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1555.696104] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1556.114192] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1556.308289] Lustre: lustre-MDT0000: new disk, initializing [ 1556.493969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1563.450246] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1577.700402] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1587.762689] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1588.287595] Lustre: lustre-OST0000: new disk, initializing [ 1588.302111] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1588.319675] Lustre: Skipped 1 previous similar message [ 1588.322655] Lustre: 54917:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1589.699771] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1589.712314] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1589.759240] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1595.973822] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1607.548324] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1610.603123] Lustre: DEBUG MARKER: create test files [ 1651.170403] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1651.177872] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1651.187904] Lustre: Skipped 12 previous similar messages [ 1654.341827] Lustre: server umount lustre-OST0000 complete [ 1659.170533] Lustre: server umount lustre-MDT0000 complete [ 1671.695716] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 1674.797713] Key type lgssc unregistered [ 1675.171500] LNet: 56551:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1675.179929] LNetError: 56551:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1675.201335] LNet: Removed LNI 192.168.202.149@tcp [ 1676.204163] Key type .llcrypt unregistered [ 1676.207113] Key type ._llcrypt unregistered [ 1706.565160] Key type ._llcrypt registered [ 1706.566820] Key type .llcrypt registered [ 1706.669160] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1707.907553] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1707.952094] alg: No test for adler32 (adler32-zlib) [ 1709.367062] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 1709.726552] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 1711.575934] Key type lgssc registered [ 1713.431021] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1722.341128] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1722.383376] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1724.078373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1729.574476] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1736.936055] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1744.483401] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1744.886868] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1748.987580] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1749.209544] Lustre: 59286:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1749.215755] Lustre: 59286:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1749.222585] Lustre: 59286:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1749.233558] Lustre: 59286:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1749.238751] Lustre: 59286:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1749.245492] Lustre: 59286:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1753.752412] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1762.587869] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1769.021814] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1775.740477] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1810.405996] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1810.445302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1810.675754] Lustre: server umount lustre-OST0000 complete [ 1815.001849] Lustre: server umount lustre-MDT0000 complete [ 1827.413739] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 1831.196972] Key type lgssc unregistered [ 1831.717426] LNet: 60915:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1831.737918] LNetError: 60915:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1831.773140] LNet: Removed LNI 192.168.202.149@tcp [ 1833.029121] Key type .llcrypt unregistered [ 1833.031445] Key type ._llcrypt unregistered [ 1870.326738] Key type ._llcrypt registered [ 1870.328939] Key type .llcrypt registered [ 1870.419723] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1871.928609] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1871.952524] alg: No test for adler32 (adler32-zlib) [ 1872.936512] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 1873.127727] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 1874.783224] Key type lgssc registered [ 1876.071381] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1884.012244] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1884.048629] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1885.641640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1890.728158] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1899.236833] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 1907.351700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1907.861890] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1910.978113] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1911.066189] Lustre: 63747:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1911.075137] Lustre: 63747:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1911.081264] Lustre: 63747:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1911.088778] Lustre: 63747:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1911.096668] Lustre: 63747:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1911.101870] Lustre: 63747:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1914.306881] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 1922.127754] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 1927.415869] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1936.870594] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1936.904766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1941.986974] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1942.803509] Lustre: server umount lustre-OST0000 complete [ 1947.546079] Lustre: server umount lustre-MDT0000 complete [ 1959.852362] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 1962.730277] Key type lgssc unregistered [ 1963.116875] LNet: 65302:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1963.128638] LNetError: 65302:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1963.146895] LNet: Removed LNI 192.168.202.149@tcp [ 1964.102910] Key type .llcrypt unregistered [ 1964.107485] Key type ._llcrypt unregistered [ 1983.057736] Key type ._llcrypt registered [ 1983.059811] Key type .llcrypt registered [ 1983.148296] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 1997.456769] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 1998.955931] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1999.100518] alg: No test for adler32 (adler32-zlib) [ 2000.143781] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 2000.307671] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 2002.025693] Key type lgssc registered [ 2003.345537] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2041.435252] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2041.466593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2042.916023] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2042.977022] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2043.175332] Lustre: lustre-MDT0000: new disk, initializing [ 2043.286358] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2043.310961] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2048.654984] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2062.838453] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2063.177261] Lustre: 68458:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2063.317520] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2063.324187] Lustre: Skipped 1 previous similar message [ 2063.471548] Lustre: lustre-MDT0001: new disk, initializing [ 2063.669301] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2063.763558] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2063.790633] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2069.512608] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2080.708733] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2087.501677] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2096.658351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2096.969634] Lustre: lustre-OST0000: new disk, initializing [ 2096.976521] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2096.982824] Lustre: 70426:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2097.120346] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2104.461246] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2106.438362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2106.448780] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2106.552277] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2114.477930] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2120.003558] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2120.254574] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2125.467333] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2125.745357] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2132.448566] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2132.449089] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2132.457736] LustreError: Skipped 1 previous similar message [ 2132.474667] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2132.477025] Lustre: Skipped 1 previous similar message [ 2137.606903] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2137.616979] Lustre: Skipped 2 previous similar messages [ 2142.177261] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2142.350232] Lustre: server umount lustre-OST0000 complete [ 2147.813696] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2147.817913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2147.831698] Lustre: Skipped 1 previous similar message [ 2147.834972] Lustre: Skipped 1 previous similar message [ 2152.928642] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2152.946518] Lustre: Skipped 1 previous similar message [ 2153.774965] Lustre: server umount lustre-MDT0000 complete [ 2157.659773] LustreError: 70425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403653 with bad export cookie 12350450743336059807 [ 2157.666966] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2157.674049] LustreError: 70425:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2158.118970] Lustre: server umount lustre-MDT0001 complete [ 2167.549679] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 19:14:20 (1786403660) [ 2180.676988] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 2230.771950] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 2243.032994] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 2283.398567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2283.655475] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2283.704693] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2283.778778] Lustre: lustre-MDT0000: new disk, initializing [ 2283.850843] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2283.897527] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2288.945941] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2300.557513] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2300.796751] Lustre: 76448:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2300.881324] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2300.888202] Lustre: Skipped 1 previous similar message [ 2301.045989] Lustre: lustre-MDT0001: new disk, initializing [ 2301.157688] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2301.192232] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2301.216548] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2305.671902] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2316.786155] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2323.632708] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2331.607522] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2331.860706] Lustre: lustre-OST0000: new disk, initializing [ 2331.864549] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2331.869981] Lustre: 78421:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2331.970848] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2339.138694] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2342.005534] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2342.014698] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2342.188956] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2350.249638] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2355.365926] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2355.602354] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2360.930311] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2361.114742] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2367.970460] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2367.988957] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2367.993111] Lustre: Skipped 1 previous similar message [ 2368.015495] Lustre: Skipped 1 previous similar message [ 2369.638503] Lustre: server umount lustre-OST0000 complete [ 2378.215521] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2378.229093] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2378.230367] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2378.243528] Lustre: Skipped 1 previous similar message [ 2382.044584] Lustre: server umount lustre-MDT0000 complete [ 2387.124932] LustreError: 76441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786403883 with bad export cookie 12350450743336060689 [ 2387.138809] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2387.143813] LustreError: 76441:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2387.658711] Lustre: server umount lustre-MDT0001 complete [ 2398.747580] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 19:18:11 (1786403891) [ 2407.234389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2407.783237] LustreError: 80833:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2407.810975] LustreError: 80833:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2407.851752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2412.952472] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2413.027681] LustreError: 80834:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2417.123524] LustreError: 80833:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2422.258667] LustreError: 80834:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2422.709322] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2429.202655] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2438.367560] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2444.750335] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2452.598620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2452.927253] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2452.931300] Lustre: Skipped 1 previous similar message [ 2459.333996] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2467.156425] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2470.563361] Lustre: 83503:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 38f2ed8f-3c77-46ca-afd6-5b2c7aac1849 at adminstrative request [ 2472.428491] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2472.447104] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2482.669252] Lustre: MGS: Client 38f2ed8f-3c77-46ca-afd6-5b2c7aac1849 (at 0@lo) reconnecting [ 2482.679448] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xab659d83fc814a83 to 0xab659d83fc814d23 [ 2482.695904] Lustre: MGC192.168.202.149@tcp: Connection restored to 0@lo (at 0@lo) [ 2509.198843] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2509.445813] Lustre: lustre-OST0001: new disk, initializing [ 2509.449699] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2509.454955] Lustre: 83900:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2515.862345] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2517.073821] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2517.079676] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2517.106465] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2526.825546] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2532.843399] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2532.843613] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2532.853202] LustreError: Skipped 1 previous similar message [ 2532.854372] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2532.854379] Lustre: Skipped 1 previous similar message [ 2532.886117] Lustre: Skipped 1 previous similar message [ 2537.322926] Lustre: server umount lustre-OST0001 complete [ 2537.957456] LustreError: 82657:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2537.982794] LustreError: 82657:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 2543.078478] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2543.105763] Lustre: Skipped 1 previous similar message [ 2548.197416] LustreError: 82655:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2548.228251] LustreError: 82655:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2548.483404] Lustre: server umount lustre-OST0000 complete [ 2556.393337] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2562.851717] Lustre: server umount lustre-MDT0000 complete [ 2567.606053] LustreError: 80814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404063 with bad export cookie 12350450743336062243 [ 2567.613619] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2567.615277] LustreError: 80814:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2568.167434] Lustre: server umount lustre-MDT0001 complete [ 2578.662336] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 19:21:11 (1786404071) [ 2580.638969] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2582.945032] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 19:21:15 (1786404075) [ 2584.558214] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2587.293600] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 19:21:19 (1786404079) [ 2589.264971] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2591.601847] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 19:21:24 (1786404084) [ 2593.666046] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2596.166497] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 19:21:28 (1786404088) [ 2598.027875] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2605.931515] loop: module loaded [ 2605.957650] loop0: detected capacity change from 0 to 4294967296 [ 2605.986092] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2608.203594] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2612.349041] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2617.260081] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2647.112669] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2647.150932] LustreError: 88028:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2647.161440] LustreError: 88028:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2647.166457] LustreError: 88028:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2647.184512] LustreError: 88028:0:(super25.c:184:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2668.076472] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 2679.776964] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 2722.365578] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2722.608549] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2722.636820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2722.697683] Lustre: lustre-MDT0000: new disk, initializing [ 2722.801940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2722.812111] Lustre: Skipped 1 previous similar message [ 2722.844731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2727.732620] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2740.105772] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2740.291394] Lustre: 90902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2740.637465] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2746.117413] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2758.082476] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2766.853585] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2776.082480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2776.473784] Lustre: 92874:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2783.378899] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2783.639843] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2784.810085] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2796.096405] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2800.632140] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2800.939866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2806.494168] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2806.757825] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2809.316462] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2809.323692] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2809.327104] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2809.327115] Lustre: Skipped 8 previous similar messages [ 2809.332622] Lustre: Skipped 3 previous similar messages [ 2809.378445] LustreError: Skipped 1 previous similar message [ 2815.416863] Lustre: server umount lustre-OST0000 complete [ 2822.628075] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2826.985644] Lustre: server umount lustre-MDT0000 complete [ 2829.810330] LustreError: 92123:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2829.832156] LustreError: 92123:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 2831.167253] LustreError: 90894:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404327 with bad export cookie 12350450743336062859 [ 2831.186982] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2831.528756] Lustre: server umount lustre-MDT0001 complete [ 2841.721918] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 19:25:34 (1786404334) [ 2850.113174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2850.911972] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2850.927610] Lustre: Skipped 2 previous similar messages [ 2855.815742] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2865.668409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2870.402601] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2879.208980] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 2885.506440] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 2893.803528] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2903.088447] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2915.243208] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 2922.343215] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2922.609574] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2927.703292] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2928.024529] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2935.399559] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2935.575519] Lustre: lustre-OST0001: new disk, initializing [ 2935.578326] Lustre: Skipped 2 previous similar messages [ 2935.581745] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2935.586298] Lustre: Skipped 3 previous similar messages [ 2935.590713] Lustre: 98656:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2941.991975] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 2945.733209] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2945.737499] Lustre: Skipped 2 previous similar messages [ 2945.741357] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2945.885244] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2953.519376] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [ 2958.701270] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2959.090525] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2963.836246] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2964.119630] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2984.936641] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3011.816063] Lustre: Failing over lustre-MDT0000 [ 3012.362843] Lustre: lustre-MDT0000: Not available for connect from 192.168.202.49@tcp (stopping) [ 3012.385619] Lustre: Skipped 7 previous similar messages [ 3012.603180] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3012.630601] Lustre: Skipped 4 previous similar messages [ 3014.383723] Lustre: server umount lustre-MDT0000 complete [ 3017.451057] LustreError: 95282:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3017.464627] LustreError: 95282:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [ 3034.591186] Lustre: 66682:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786404513/real 1786404513] req@00000000a1c57459 x1873179832512128/t0(0) o400->MGC192.168.202.149@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786404529 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3034.634902] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3040.375572] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3040.378593] LDISKFS-fs (dm-0): recovery complete [ 3040.388632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3045.302581] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3049.199716] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3049.201128] Lustre: 95280:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3050.472790] Lustre: 95285:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3050.488488] Lustre: 95285:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 1 previous similar message [ 3050.510724] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3050.616290] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3050.945056] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3050.985164] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3051.296044] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3051.306526] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 3051.632423] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3051.638863] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 1 previous similar message [ 3051.660460] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3051.672232] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 7 previous similar messages [ 3052.333441] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3052.347347] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 3052.665357] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3052.673026] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 2 previous similar messages [ 3053.663692] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3053.685639] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 5 previous similar messages [ 3054.369869] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3054.382665] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 3054.719205] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3054.730018] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 5 previous similar messages [ 3057.968717] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3057.992144] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 12 previous similar messages [ 3058.645119] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3058.654119] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 3058.983205] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3058.991199] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 12 previous similar messages [ 3066.032291] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3066.065100] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 24 previous similar messages [ 3066.712556] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3066.736411] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 3067.055193] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3067.067418] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 24 previous similar messages [ 3082.117704] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3082.144516] Lustre: 101512:0:(ldlm_lib.c:2071:extend_recovery_timer()) Skipped 49 previous similar messages [ 3082.800044] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3082.813091] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 3083.151151] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3083.159498] LustreError: 101512:0:(ldlm_lib.c:2684:replay_request_or_update()) Skipped 49 previous similar messages [ 3109.071284] Lustre: 101512:0:(ldlm_lib.c:2380:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3109.090240] Lustre: 101512:0:(ldlm_lib.c:2390:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3109.097637] Lustre: 101512:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5b7121b3-4f01-4014-beac-14d03b1a277e@192.168.202.49@tcp [ 3109.104883] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3109.108928] Lustre: 101512:0:(ldlm_lib.c:1900:abort_req_replay_queue()) @@@ aborted: req@00000000ad65841b x1873179815516288/t0(8589934778) o35->5b7121b3-4f01-4014-beac-14d03b1a277e@192.168.202.49@tcp:625/0 lens 392/0 e 0 to 0 dl 1786404615 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3109.132611] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3109.140423] Lustre: lustre-MDT0000: Denying connection for new client 5b7121b3-4f01-4014-beac-14d03b1a277e (at 192.168.202.49@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3109.145531] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3109.187702] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3109.192473] Lustre: Skipped 2 previous similar messages [ 3109.195179] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3109.221077] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3109.224303] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3127.288475] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3127.302257] Lustre: Skipped 1 previous similar message [ 3132.392928] LustreError: 97108:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3132.415821] LustreError: 97108:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 41 previous similar messages [ 3132.512155] Lustre: server umount lustre-OST0000 complete [ 3142.636332] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3142.649490] Lustre: Skipped 9 previous similar messages [ 3143.365290] Lustre: server umount lustre-OST0001 complete [ 3153.813386] Lustre: server umount lustre-MDT0000 complete [ 3159.212964] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404655 with bad export cookie 12350450743336114078 [ 3159.224913] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3159.242484] LustreError: 95265:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 3159.969136] Lustre: server umount lustre-MDT0001 complete [ 3173.004670] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 19:31:05 (1786404665) [ 3181.669456] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3181.677809] Lustre: *** cfs_fail_loc=197, val=0*** [ 3197.215844] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3202.339519] LustreError: 103488:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.149@tcp: failed processing log, type 4: rc = -110 [ 3207.467850] Lustre: *** cfs_fail_loc=197, val=0*** [ 3207.470588] Lustre: Skipped 65 previous similar messages [ 3228.064671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3228.068677] Lustre: Skipped 4 previous similar messages [ 3233.570433] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3240.742887] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3243.249240] Lustre: server umount lustre-OST0000 complete [ 3251.933510] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 19:32:25 (1786404745) [ 3271.793632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3272.191815] LustreError: 105108:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3272.206381] LustreError: 105108:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [ 3276.754780] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3287.715735] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3292.304631] LustreError: 105139:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3292.316268] LustreError: 105139:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3292.322136] LustreError: 105139:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3292.706070] Lustre: server umount lustre-MDT0000 complete [ 3327.623041] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 3338.018760] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 3378.572950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3378.845842] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3378.882431] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3378.996526] Lustre: lustre-MDT0000: new disk, initializing [ 3379.131859] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3383.146660] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3395.989922] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3396.134534] Lustre: 109314:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3396.547424] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3402.663378] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3413.868251] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3420.061172] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3428.465277] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3428.888391] Lustre: 111284:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3435.642103] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3436.329379] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3444.982882] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3450.063569] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3450.240713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3455.771196] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3456.041767] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3462.113732] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3462.129307] Lustre: Skipped 5 previous similar messages [ 3462.137732] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3462.144995] Lustre: Skipped 5 previous similar messages [ 3472.353551] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3472.629721] Lustre: server umount lustre-OST0000 complete [ 3482.931547] Lustre: server umount lustre-MDT0000 complete [ 3486.575549] LustreError: 109306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786404982 with bad export cookie 12350450743336124753 [ 3486.579305] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3486.583626] LustreError: 109306:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3487.091897] Lustre: server umount lustre-MDT0001 complete [ 3497.901280] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 19:36:30 (1786404990) [ 3507.159403] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 3511.242470] Key type lgssc unregistered [ 3511.526677] LNet: 113772:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3511.540514] LNetError: 113772:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3511.556820] LNet: Removed LNI 192.168.202.149@tcp [ 3512.426886] Key type .llcrypt unregistered [ 3512.428640] Key type ._llcrypt unregistered [ 3535.329691] Key type ._llcrypt registered [ 3535.334594] Key type .llcrypt registered [ 3535.456868] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 3553.992142] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 3555.988731] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3556.033092] alg: No test for adler32 (adler32-zlib) [ 3557.069468] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 3557.327433] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 3559.143195] Key type lgssc registered [ 3561.098328] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3613.419884] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3613.473222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3615.017062] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3615.086740] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3615.229108] Lustre: lustre-MDT0000: new disk, initializing [ 3615.390752] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3615.430795] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3621.920923] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3634.440307] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3634.910996] Lustre: lustre-OST0000: new disk, initializing [ 3634.919722] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3634.928643] Lustre: Skipped 1 previous similar message [ 3634.934550] Lustre: 117949:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3635.042701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3642.828918] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3643.041068] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3643.066503] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3643.122856] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3654.476167] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3654.778261] Lustre: lustre-OST0001: new disk, initializing [ 3654.782781] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3654.789923] Lustre: 118811:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3654.966346] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3661.484722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3661.497218] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3661.527272] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3663.278388] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3676.642299] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3676.649227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3676.658589] Lustre: Skipped 1 previous similar message [ 3676.680549] Lustre: Skipped 1 previous similar message [ 3679.910804] Lustre: server umount lustre-MDT0000 complete [ 3686.603672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3691.251609] LustreError: 117137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405187 with bad export cookie 3557302510498762270 [ 3691.258904] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3691.265337] LustreError: 117137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3691.436915] Lustre: server umount lustre-OST0000 complete [ 3695.368842] Lustre: server umount lustre-OST0001 complete [ 3704.727179] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 19:39:57 (1786405197) [ 3729.580985] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 19:40:22 (1786405222) [ 3739.684976] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 3765.036725] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 19:40:57 (1786405257) [ 3808.868409] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 3818.471702] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 3863.538331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3863.899416] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3863.927364] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3863.987724] Lustre: lustre-MDT0000: new disk, initializing [ 3864.078490] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3864.100585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3869.327546] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3880.724927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3880.828789] Lustre: 126568:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3880.908079] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3880.912973] Lustre: Skipped 1 previous similar message [ 3880.991305] Lustre: lustre-MDT0001: new disk, initializing [ 3881.116843] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3881.162721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3881.178131] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3885.792139] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3896.692186] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 3903.024171] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 3911.987749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3912.331481] Lustre: lustre-OST0000: new disk, initializing [ 3912.338272] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3912.349268] Lustre: 128539:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3912.498219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3918.928767] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3918.937466] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3919.087925] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3919.558412] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3930.526754] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 3935.898441] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3936.153142] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3940.979242] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3941.190283] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3944.931729] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3944.934815] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3944.938157] LustreError: Skipped 1 previous similar message [ 3944.940048] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3944.980024] Lustre: Skipped 1 previous similar message [ 3949.383457] Lustre: server umount lustre-OST0000 complete [ 3955.175741] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3955.183867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3955.205581] Lustre: Skipped 1 previous similar message [ 3955.223400] Lustre: Skipped 2 previous similar messages [ 3960.298703] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3960.317041] Lustre: Skipped 1 previous similar message [ 3960.629774] Lustre: server umount lustre-MDT0000 complete [ 3965.382220] LustreError: 128549:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405461 with bad export cookie 3557302510498763495 [ 3965.390611] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3965.871741] Lustre: server umount lustre-MDT0001 complete [ 3975.656350] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 19:44:28 (1786405468) [ 3983.008761] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3983.471520] LustreError: 130952:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3983.493980] LustreError: 130952:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 3983.580906] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3987.827930] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 3988.961854] LustreError: 130953:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3994.090347] LustreError: 130952:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3997.111771] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4001.975927] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4010.712293] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4018.032153] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4026.747876] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4027.148302] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4027.160425] Lustre: Skipped 1 previous similar message [ 4034.167989] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4041.830756] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4046.880227] Lustre: *** cfs_fail_loc=159, val=0*** [ 4062.559507] Lustre: lustre-MDT0000: Client 272c21b3-d9e6-47f5-8584-0e878fefb9d4 (at 192.168.202.49@tcp) reconnecting [ 4064.828418] Lustre: *** cfs_fail_loc=159, val=0*** [ 4064.831116] Lustre: Skipped 5 previous similar messages [ 4080.444237] Lustre: lustre-MDT0000: Client 272c21b3-d9e6-47f5-8584-0e878fefb9d4 (at 192.168.202.49@tcp) reconnecting [ 4085.729380] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4085.742777] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4085.764402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4087.646891] Lustre: server umount lustre-OST0000 complete [ 4093.921522] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4093.932650] Lustre: Skipped 1 previous similar message [ 4093.933566] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4093.950100] Lustre: Skipped 1 previous similar message [ 4098.408780] Lustre: server umount lustre-MDT0000 complete [ 4099.041265] LustreError: 133551:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4099.051440] LustreError: 133551:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4103.021365] LustreError: 130935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405598 with bad export cookie 3557302510498764377 [ 4103.028789] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4103.035140] LustreError: 130935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 4112.883470] Lustre: server umount lustre-MDT0001 complete [ 4129.961827] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 4134.315631] Key type lgssc unregistered [ 4134.929314] LNet: 134736:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4134.965903] LNetError: 134736:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4134.983220] LNet: Removed LNI 192.168.202.149@tcp [ 4136.385714] Key type .llcrypt unregistered [ 4136.387459] Key type ._llcrypt unregistered [ 4165.290562] Key type ._llcrypt registered [ 4165.292740] Key type .llcrypt registered [ 4165.423625] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 4166.803803] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4166.901091] alg: No test for adler32 (adler32-zlib) [ 4168.087787] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 4168.392791] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 4170.119280] Key type lgssc registered [ 4171.463506] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4179.611787] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4179.626855] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4181.206343] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4185.404952] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4191.715860] LustreError: 136157:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4191.725447] LustreError: 136157:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4193.820957] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4194.135575] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4199.114898] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4206.977543] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4213.300668] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4220.755251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4221.225211] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4226.388876] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4227.884615] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4237.493067] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4248.333376] Lustre: *** cfs_fail_loc=159, val=0*** [ 4264.755187] Lustre: lustre-MDT0000: Client e4231923-24cf-4193-97c5-b2d7b91a2a97 (at 192.168.202.49@tcp) reconnecting [ 4267.421514] Lustre: *** cfs_fail_loc=159, val=0*** [ 4283.727438] Lustre: lustre-MDT0001: Client e4231923-24cf-4193-97c5-b2d7b91a2a97 (at 192.168.202.49@tcp) reconnecting [ 4286.343929] Lustre: *** cfs_fail_loc=159, val=0*** [ 4286.345769] Lustre: Skipped 3 previous similar messages [ 4302.600614] Lustre: lustre-MDT0001: Client e4231923-24cf-4193-97c5-b2d7b91a2a97 (at 192.168.202.49@tcp) reconnecting [ 4318.542796] Lustre: 139219:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4321.436210] Lustre: *** cfs_fail_loc=159, val=0*** [ 4321.437838] Lustre: Skipped 4 previous similar messages [ 4338.027983] Lustre: lustre-MDT0000: Client 8506e54e-5db2-4c84-a891-0965f1e3a82d (at 192.168.202.49@tcp) reconnecting [ 4340.319883] Lustre: *** cfs_fail_loc=159, val=0*** [ 4340.328136] Lustre: Skipped 14 previous similar messages [ 4356.927633] Lustre: lustre-MDT0000: Client 8506e54e-5db2-4c84-a891-0965f1e3a82d (at 192.168.202.49@tcp) reconnecting [ 4358.876941] Lustre: 139514:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4366.822135] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4366.825798] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4366.841851] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4366.857153] Lustre: Skipped 1 previous similar message [ 4371.938177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4371.942326] Lustre: Skipped 1 previous similar message [ 4372.176423] Lustre: server umount lustre-OST0000 complete [ 4378.613196] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4378.633954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4378.654050] Lustre: Skipped 1 previous similar message [ 4378.659299] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4382.177932] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4382.178721] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4383.616040] Lustre: server umount lustre-MDT0000 complete [ 4387.302105] LustreError: 136157:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4387.880775] LustreError: 136138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786405883 with bad export cookie 13801704533122943425 [ 4387.888215] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4387.895712] LustreError: 136138:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4388.417957] Lustre: server umount lustre-MDT0001 complete [ 4401.722442] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 4404.926424] Key type lgssc unregistered [ 4405.163828] LNet: 140568:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4405.170457] LNetError: 140568:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4406.180847] LNet: Removed LNI 192.168.202.149@tcp [ 4407.407091] Key type .llcrypt unregistered [ 4407.409530] Key type ._llcrypt unregistered [ 4438.542954] Key type ._llcrypt registered [ 4438.551957] Key type .llcrypt registered [ 4438.746690] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 4440.580914] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4440.678770] alg: No test for adler32 (adler32-zlib) [ 4441.710355] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 4441.921876] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 4443.551217] Key type lgssc registered [ 4444.829936] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4454.849873] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4454.884028] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4456.573953] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4462.187188] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4466.174252] LustreError: 141985:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4466.209553] LustreError: 141985:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4471.273478] LustreError: 141984:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4472.050237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4472.279893] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4478.029155] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4487.750181] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4495.768620] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4504.330110] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4504.710888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4511.611380] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4518.075068] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4521.174481] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4526.309700] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4532.399588] Lustre: 144572:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4537.090696] Lustre: 144621:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4538.939182] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4540.039494] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4547.459847] Lustre: 144768:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4548.719164] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4550.342477] Lustre: 144876:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4557.292122] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4557.302974] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4557.308474] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4557.358645] Lustre: Skipped 1 previous similar message [ 4562.403591] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4562.422023] Lustre: Skipped 2 previous similar messages [ 4563.201696] Lustre: server umount lustre-OST0000 complete [ 4567.541195] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4567.549298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4567.556521] Lustre: Skipped 1 previous similar message [ 4567.574753] Lustre: Skipped 1 previous similar message [ 4572.641721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4572.654765] Lustre: Skipped 1 previous similar message [ 4573.579596] Lustre: server umount lustre-MDT0000 complete [ 4576.893184] LustreError: 141966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786406072 with bad export cookie 15472555425106424818 [ 4576.898543] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4576.903150] LustreError: 141966:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4577.158810] Lustre: server umount lustre-MDT0001 complete [ 4587.771335] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 4590.319912] Key type lgssc unregistered [ 4590.539974] LNet: 145932:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4590.547114] LNetError: 145932:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4591.589633] LNet: Removed LNI 192.168.202.149@tcp [ 4592.486456] Key type .llcrypt unregistered [ 4592.488119] Key type ._llcrypt unregistered [ 4618.146617] Key type ._llcrypt registered [ 4618.148329] Key type .llcrypt registered [ 4618.286171] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 4620.227280] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4620.305911] alg: No test for adler32 (adler32-zlib) [ 4621.447418] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 4621.739594] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 4623.431249] Key type lgssc registered [ 4624.475726] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4632.609733] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4632.633026] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4634.418715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4639.691768] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4644.837110] LustreError: 147348:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4644.850995] LustreError: 147348:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4649.371215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4650.039471] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4655.667894] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4665.845241] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4671.346822] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4679.174492] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4679.738145] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4684.818013] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4687.556470] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4695.089367] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4695.565452] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4702.253367] Lustre: *** cfs_fail_loc=159, val=0*** [ 4718.890253] Lustre: lustre-MDT0000: Client c5d781ef-5af9-4fd9-bf23-b0f76457effe (at 192.168.202.49@tcp) reconnecting [ 4725.728019] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4725.730888] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4725.739155] LustreError: Skipped 1 previous similar message [ 4725.742136] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4725.781202] Lustre: Skipped 1 previous similar message [ 4730.852846] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4730.858609] Lustre: Skipped 2 previous similar messages [ 4731.894193] Lustre: server umount lustre-OST0000 complete [ 4735.969831] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4735.981153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4735.986464] Lustre: Skipped 1 previous similar message [ 4736.018704] Lustre: Skipped 1 previous similar message [ 4741.089422] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4741.092882] Lustre: Skipped 1 previous similar message [ 4746.211156] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4746.220835] Lustre: Skipped 1 previous similar message [ 4750.303747] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4750.586352] Lustre: server umount lustre-MDT0000 complete [ 4751.344063] LustreError: 147349:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4751.365174] LustreError: 147349:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4754.434420] LustreError: 147328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786406250 with bad export cookie 14825861283763777777 [ 4754.437912] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4754.456824] LustreError: 147328:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4754.854397] Lustre: server umount lustre-MDT0001 complete [ 4766.301061] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 4769.575853] Key type lgssc unregistered [ 4769.942565] LNet: 151016:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4769.946484] LNetError: 151016:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4770.987140] LNet: Removed LNI 192.168.202.149@tcp [ 4772.304236] Key type .llcrypt unregistered [ 4772.306646] Key type ._llcrypt unregistered [ 4804.330763] Key type ._llcrypt registered [ 4804.334629] Key type .llcrypt registered [ 4804.450392] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 4805.463346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4805.512447] alg: No test for adler32 (adler32-zlib) [ 4806.831207] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 4807.074261] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 4808.783239] Key type lgssc registered [ 4809.870627] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4817.796531] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4817.823790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4819.361751] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4824.537954] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4829.667112] LustreError: 152435:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4829.695458] LustreError: 152435:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 4834.784990] LustreError: 152434:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4835.822704] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4836.491755] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4842.132203] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4852.270665] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 4859.670240] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 4868.066340] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4868.562508] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4876.184683] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 4881.901554] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4884.047785] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 4890.112906] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4910.746747] Lustre: lustre-MDT0001: evicting 54dfacf4-7ed3-4ae7-9450-7988b440d0dd (at 192.168.202.49@tcp) by administrative request [ 4943.328274] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4943.338203] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4943.361609] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4946.409288] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4946.413217] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4946.416761] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4951.522415] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4951.543573] Lustre: Skipped 2 previous similar messages [ 4956.128101] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4956.304783] Lustre: server umount lustre-OST0000 complete [ 4961.763657] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4961.771447] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4961.778879] Lustre: Skipped 1 previous similar message [ 4961.793507] Lustre: Skipped 1 previous similar message [ 4966.456868] Lustre: server umount lustre-MDT0000 complete [ 4966.887219] LustreError: 153565:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4966.910311] LustreError: 153565:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 4970.643822] LustreError: 154287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786406466 with bad export cookie 4302934668908352194 [ 4970.650467] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4970.654971] LustreError: 154287:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4970.995364] Lustre: server umount lustre-MDT0001 complete [ 4982.158772] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 4984.763962] Key type lgssc unregistered [ 4985.023795] LNet: 156517:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4985.031903] LNetError: 156517:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4985.044533] LNet: Removed LNI 192.168.202.149@tcp [ 4985.840974] Key type .llcrypt unregistered [ 4985.847086] Key type ._llcrypt unregistered [ 5092.291918] Key type ._llcrypt registered [ 5092.296914] Key type .llcrypt registered [ 5092.465953] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 5094.277724] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5094.453547] alg: No test for adler32 (adler32-zlib) [ 5095.878648] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 5096.121220] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 5097.943208] Key type lgssc registered [ 5099.608197] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5109.242771] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5109.291528] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5111.015485] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5117.222054] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5120.483860] LustreError: 159019:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5120.519417] LustreError: 159019:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5125.609227] LustreError: 159020:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5128.330152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5128.859689] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5132.738071] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5143.352333] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5149.547390] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5158.834644] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5159.538135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5165.540233] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5172.998448] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5173.883835] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5181.059566] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5194.723942] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5194.747338] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5194.779932] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5196.780511] Lustre: server umount lustre-OST0000 complete [ 5201.891385] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5201.912664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5201.926441] Lustre: Skipped 1 previous similar message [ 5207.011652] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5207.018134] Lustre: Skipped 1 previous similar message [ 5210.315964] Lustre: server umount lustre-MDT0000 complete [ 5211.106116] LustreError: 159015:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5211.136658] LustreError: 159015:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 5215.446691] LustreError: 159000:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786406711 with bad export cookie 9513878757110802696 [ 5215.489711] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5216.304957] Lustre: server umount lustre-MDT0001 complete [ 5232.639927] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 5236.131749] Key type lgssc unregistered [ 5236.627778] LNet: 162587:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5236.632853] LNetError: 162587:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5236.655951] LNet: Removed LNI 192.168.202.149@tcp [ 5237.602963] Key type .llcrypt unregistered [ 5237.606339] Key type ._llcrypt unregistered [ 5270.421389] Key type ._llcrypt registered [ 5270.423960] Key type .llcrypt registered [ 5270.594345] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 5272.194506] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5272.301498] alg: No test for adler32 (adler32-zlib) [ 5273.497599] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 5273.816505] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 5275.535251] Key type lgssc registered [ 5276.955871] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5297.317111] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 20:06:29 (1786406789) [ 5318.273770] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 5330.680720] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 5374.309718] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5374.343720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5375.851760] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5375.914484] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5376.080661] Lustre: lustre-MDT0000: new disk, initializing [ 5376.525498] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5376.564233] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5381.671462] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5394.509127] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5394.801851] Lustre: 166855:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5394.970443] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5394.983044] Lustre: Skipped 1 previous similar message [ 5395.136180] Lustre: lustre-MDT0001: new disk, initializing [ 5395.280480] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5395.328062] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5395.341301] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5400.491912] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5412.268709] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5419.944621] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5428.841433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5429.347861] Lustre: lustre-OST0000: new disk, initializing [ 5429.357786] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5429.368846] Lustre: 168824:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5429.511050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5437.656090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5437.680577] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5437.942994] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5438.562653] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5454.349311] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5468.643951] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5468.648105] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5468.649301] LustreError: Skipped 1 previous similar message [ 5468.649846] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5468.680151] Lustre: Skipped 1 previous similar message [ 5472.180646] Lustre: server umount lustre-OST0000 complete [ 5477.344359] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5477.350209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5477.367311] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5477.373527] Lustre: Skipped 1 previous similar message [ 5478.881440] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5478.882017] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5478.900579] Lustre: Skipped 1 previous similar message [ 5484.000691] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5484.010983] Lustre: Skipped 1 previous similar message [ 5488.100111] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5488.113016] Lustre: Skipped 1 previous similar message [ 5491.167360] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5491.513336] Lustre: server umount lustre-MDT0000 complete [ 5495.223883] LustreError: 166845:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786406991 with bad export cookie 7741404933355654305 [ 5495.233138] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5495.562506] Lustre: server umount lustre-MDT0001 complete [ 5508.469986] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 5511.596434] Key type lgssc unregistered [ 5511.966251] LNet: 170733:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5511.987631] LNetError: 170733:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5512.010319] LNet: Removed LNI 192.168.202.149@tcp [ 5513.088442] Key type .llcrypt unregistered [ 5513.091401] Key type ._llcrypt unregistered [ 5544.083098] Key type ._llcrypt registered [ 5544.087285] Key type .llcrypt registered [ 5544.232408] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 5545.292572] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5545.330681] alg: No test for adler32 (adler32-zlib) [ 5546.314274] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 5546.550938] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 5548.327506] Key type lgssc registered [ 5549.416795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5557.764645] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5557.786434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5559.240966] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5564.599549] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5569.515125] LustreError: 172157:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5569.542466] LustreError: 172157:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 5574.626812] LustreError: 172158:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5575.313224] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5575.771252] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5580.256531] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5588.892107] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 5594.696612] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 5601.487419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5601.798978] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5604.925288] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5608.232875] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 5615.368186] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 5625.155251] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5625.175343] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5625.196688] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5625.209315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.233288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5626.291278] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5626.301825] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5626.323945] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5626.337114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.358544] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5627.433720] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5627.443763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5627.451893] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5627.455694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.465349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5628.361254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.388941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.249877] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5630.257660] Lustre: Skipped 2 previous similar messages [ 5630.273412] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5630.283655] Lustre: Skipped 2 previous similar messages [ 5630.287802] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5630.293255] Lustre: Skipped 2 previous similar messages [ 5630.297736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.309432] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5630.315830] Lustre: Skipped 2 previous similar messages [ 5631.105926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.841573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.974512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.135132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.141378] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5635.145087] Lustre: Skipped 4 previous similar messages [ 5635.158654] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5635.167618] Lustre: Skipped 4 previous similar messages [ 5635.171760] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5635.177176] Lustre: Skipped 4 previous similar messages [ 5635.182986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.200183] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5635.208590] Lustre: Skipped 4 previous similar messages [ 5636.006600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.033087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.035330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.888672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.889771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.617467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.428085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.304692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.197040] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5643.203640] Lustre: Skipped 8 previous similar messages [ 5643.213869] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5643.224161] Lustre: Skipped 8 previous similar messages [ 5643.229935] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5643.234847] Lustre: Skipped 8 previous similar messages [ 5643.238640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.247384] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5643.251708] Lustre: Skipped 8 previous similar messages [ 5644.036401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.822379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.446353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.365229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.227242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.144122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.034894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.039517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.002395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.939424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.810514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.950708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.898337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.790591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.748322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.703769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.708441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.640961] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5659.649987] Lustre: Skipped 17 previous similar messages [ 5659.682453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5659.712666] Lustre: Skipped 17 previous similar messages [ 5659.717839] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5659.724663] Lustre: Skipped 17 previous similar messages [ 5659.733146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.755273] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5659.765751] Lustre: Skipped 17 previous similar messages [ 5660.741067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.643083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.654896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.517217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.606884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.581844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.662617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.712893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.005848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.220552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.414495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.692613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.018645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.351471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.674209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.052536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.415656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.467076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.439721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.593140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.793741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.954446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.094082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.138645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.223812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.001922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.146882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.032755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.164559] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5692.172626] Lustre: Skipped 28 previous similar messages [ 5692.192902] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5692.201442] Lustre: Skipped 28 previous similar messages [ 5692.205057] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5692.208794] Lustre: Skipped 28 previous similar messages [ 5692.213054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.222339] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5692.237532] Lustre: Skipped 28 previous similar messages [ 5693.260604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.519903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.942978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.283369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.630406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.721552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.655540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.572315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.493308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.497450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.706043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.614143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.584569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.681667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.748201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.815677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.958341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.974230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.011617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.097452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.288225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.183538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.196057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.232909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.070673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.239896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.258639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.372113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.471625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.621739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.800769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.850938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.862395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.121333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.486952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.623095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.555619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.621812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.467952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.699832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.821714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.054836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.968042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.193378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.215273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.244445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.671646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.002415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.099364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.995916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.263639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.253493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.684045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.805348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.890112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.079121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.126675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.232478] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5756.239214] Lustre: Skipped 57 previous similar messages [ 5756.260973] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5756.269114] Lustre: Skipped 57 previous similar messages [ 5756.275762] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5756.278750] Lustre: Skipped 57 previous similar messages [ 5756.294346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.312659] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5756.330357] Lustre: Skipped 57 previous similar messages [ 5757.485735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.095624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.398545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.574841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.807560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.046683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.313985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.643597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.797658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.033981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.985037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.272775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.274176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.314331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.511689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.659452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.711750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.647370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.631508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.613266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.789213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.773709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.701748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.789670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.828660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.899402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.723324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.963770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.311362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.344213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.439949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.441962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.455389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.349615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.344713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.315349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.616486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.588773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.786972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.857175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.062200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.993719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.159984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.168282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.082960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.178352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.370299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.533049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.676344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.739107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.870085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.960444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.022024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.030797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.185347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.337568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.313600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.411489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.673657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.770487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.987406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.120874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.358976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.497993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.695958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.617819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.706687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.991782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.091621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.302353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.288630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.436717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.355687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.467800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.530583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.697974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.893574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.252665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.750928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.898745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.084523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.084333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.265115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.388822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.690224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.063894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.449134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.650453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.983792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.017557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.263919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.789573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.241455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.524044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.827780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.083080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.181487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.247426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.553859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.857845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.067826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.088830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.374836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.330976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.531760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.879929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.976510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.123595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.535559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.966675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.997460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.157555] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5885.160518] Lustre: Skipped 111 previous similar messages [ 5885.181870] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5885.192830] Lustre: Skipped 111 previous similar messages [ 5885.197096] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5885.201493] Lustre: Skipped 111 previous similar messages [ 5885.205504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.212782] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5885.218114] Lustre: Skipped 111 previous similar messages [ 5886.242700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.535835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.836610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.093113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.174706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.152572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.329180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.400513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.584973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.851234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.827369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.092353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.276964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.763440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.044387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.309660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.577871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.745466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.894403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.298392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.527492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.632100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.767432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.829070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.010819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.224476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.417421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.601685] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5918.609557] LustreError: 172177:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5918.649424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.879170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.161107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.519135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.706869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.850274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.224480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.312107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.265562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.202851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.264438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.278205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.409128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.756579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.811895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.903657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.139857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.101441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.062782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.083850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.882432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.804039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.851252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.859776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.873151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.967203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.973520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.959574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.896787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.664875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.622292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.793230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.842334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.869688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.010971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.982331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.025887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.979896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.133702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.128565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.503843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.819356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.850919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.831975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.952763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.275706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.384877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.232053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.251655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.453506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.749593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.711882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.888348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.275508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.159921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.979423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.667807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.531486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.491956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.604802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.598456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.607432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.462911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.745950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.709290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.537302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.584743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.640261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.499100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.748681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.995985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.818540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.873336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.025721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.194789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.926600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.332882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.671831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.572200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.673426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.137073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.900194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.758959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.012851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.449376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.496308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.789360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.863569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.875423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.116326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.120648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.226967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.400572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.478679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.514313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.718172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.938495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.193069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.405037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.644897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.863339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.944132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.073443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.105377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.228103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.308765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.262520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.161347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.888816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.799921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.589207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.616764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.569596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.611115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.570816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.585637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.402291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.398565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.572449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.687497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.608217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.798483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.912474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.728203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.821541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.645294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.736645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.546824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.411860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.353525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.421525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.538081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.475941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.480419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.578350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.520931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.362762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.243478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.208511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.239157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.108377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.973618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.699705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.701439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.888798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.952660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.468550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.521906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.569633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.356912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.209035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.074946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.876931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.618797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.358743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.352033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.242817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.119172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.969128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.955433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.873682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.769722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.642420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.549066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.467316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.602354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.508715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.446857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.370522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.526065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.963968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.020271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.457042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.631437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.880570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.869317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.008201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.044255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.078208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.254945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.210300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.188589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.062792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.106811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.216503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.195847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.564564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.700820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.731353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.760730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.528435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.339314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.323629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.421766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.437829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.512689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.303680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.122127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.024221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.074242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.918285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.810743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.778859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.677974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.572617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.454247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.270881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.200955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.939679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.633875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.269073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.226838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.978927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.030796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.783320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.853494] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6141.863227] Lustre: Skipped 242 previous similar messages [ 6141.877493] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6141.891191] Lustre: Skipped 242 previous similar messages [ 6141.901960] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6141.914663] Lustre: Skipped 242 previous similar messages [ 6141.936059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.951234] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6141.961227] Lustre: Skipped 242 previous similar messages [ 6142.792205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.720588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.471711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.277933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.216143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.137232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.114433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.960554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.730272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.787967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.779221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.016859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.013842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.182234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.153333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.362226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.241646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.252368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.335890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.651962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.764075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.790301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.751813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.629208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.493061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.916670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.370256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.862449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.229256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.494757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.844351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.816572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.902498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.861646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.940422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.186363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.471931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.559447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.640900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.636266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.661925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.362869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.456125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.540923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.989606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.014203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.960538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.914939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.034440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.857842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.000803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.775148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.726305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.686619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.689467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.478656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.517859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.294530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.097592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.005685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.915653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.794649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.703367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.748177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.860302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.032372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.320605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.278182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.173355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.996123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.888199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.661405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.728378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.613618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.554976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.383929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.292665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.118983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.312395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.300939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.330480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.250840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.309381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.247281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.151771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.108811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.228781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.294386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.411404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.460713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.429793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.319714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.245325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.330208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.286089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.477066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.660479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.908484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.981695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.335725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.332082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.624943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.958476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.249759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.697695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.889963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.062992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.079357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.272787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.667219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.083630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.083547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.459278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.384157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.514911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.590800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.714673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.640770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.692542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.436975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.557951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.749876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.739538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.763233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.626206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.572465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.385779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.386330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.168820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.042612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.953510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.782721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.909401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.941505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.921539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.865931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.837660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.921379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.026985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.072981] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6287.091231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.091533] LustreError: 172177:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6288.048491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.823614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.683886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.691331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.611588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.651749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.771423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.915441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.170550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.065357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.349876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.721102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.964197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.019790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.285060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.328073] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6304.335067] LustreError: 172177:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6304.347635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.317738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.585224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.684331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.760700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.917508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.925230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.075810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.102306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.359250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.358161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.348196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.388406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.245650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.120092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.936375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.791184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.700767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.672564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.621299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.684985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.887959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.649781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.497081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.310548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.250474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.287430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.278864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.272064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.378446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.403614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.174302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.160409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.328454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.252063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.368071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.311811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.256906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.074670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.932691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.766188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.536632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.185820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.151969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.971775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.877045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.815177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.659822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.506496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.312508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.123451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.918931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.826086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.952674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.810518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.557178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.442669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.389887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.247418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.089313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.100425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.083367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.068311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.143732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.146941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.148259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.259441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.126490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.987701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.857604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.931557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.842040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.797407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.802541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.757296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.729153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.743972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.615383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.585643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.534393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.461758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.325931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.291330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.154206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.009296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.780264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.606198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.395407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.308496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.196434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.205203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.025771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.998919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.945239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.939921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.838315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.964515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.044711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.749442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.696868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.470843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.635622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.017190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.036753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.125847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.382537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.332021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.555063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.545132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.821723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.030548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.281094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.732082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.076301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.343424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.596063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.911561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.940676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.246573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.336671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.663302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.278914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.393453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.817318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.026070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.162721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.322326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.661354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.534771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.651556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.051852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.377991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.637576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.645835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.634704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.580861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.734231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.715407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.739527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.829748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.776962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.631594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.614919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.139150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.618689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.890628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.159502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.985493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.965896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.727189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.709109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.589164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.615600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.606241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.445949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.176471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.338438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.313893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.436186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.273283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.081280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.858678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.897755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.805261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.853635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.732116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.639889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.505380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.508632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.516218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.383050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.227893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.184069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.265395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.304835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.580322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.650295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.828610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.905556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.985978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.024887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.857226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.813414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.689401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.602811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.422909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.203287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.156635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.104619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.855434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.768242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.817084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.563977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.335462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.249055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.117545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.933503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.821160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.616599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.394906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.256299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.036947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.925885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.722635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.721262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.736849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.718375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6509.545456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.492811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.487598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.456892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.440910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.248527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.098650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.023889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.843568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.787036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.553477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.453239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.377024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.435869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.314535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.114336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.111941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.125097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.978620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.926628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.734950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.697529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.525851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.458558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.250521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.972200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.900183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.929504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.102644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.900600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.976871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.921720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.908820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.813860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.685531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.638930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.789047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.709255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.798613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.752576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.723787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.592574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.776832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.658101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.917799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.804535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.089407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6554.079742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.506553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.518883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.431422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.482705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.630642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.681078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.548349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.601508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.515187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.719612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.117542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.315580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.596420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.811236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.872309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.041114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.142625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.158832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.209596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.200755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.425747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.496974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.528931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.807726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.494506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.590247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.505492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.947745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.110835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.057922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.087862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.208748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6591.200203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.241667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.199319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.091628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.858326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.727667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.709891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.537030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.497601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.402429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.367683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.204419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.099708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.006130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.930377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.706530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.662786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.638295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6607.563175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.449993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.604759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.469785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.656717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.379310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.317381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.404344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.253919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.392391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.631204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.754177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.912435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.355687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.169131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6623.349117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.594074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.577155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.098723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.214384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.831274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.007502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6631.889693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.303058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.800969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.463930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6638.113781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.635626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.305804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6642.713788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.964416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.156218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6646.534136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.890240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.712867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.147750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.773557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.230229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.433809] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6654.442603] Lustre: Skipped 498 previous similar messages [ 6654.480465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6654.507812] Lustre: Skipped 498 previous similar messages [ 6654.512439] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6654.520481] Lustre: Skipped 498 previous similar messages [ 6654.530189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.540776] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6654.556861] Lustre: Skipped 498 previous similar messages [ 6655.538036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6656.989136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.309444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.661837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6660.704510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.786636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.541082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.326847] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6663.338900] LustreError: 172177:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6663.346981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.962501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.429220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.337619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6668.435685] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6668.445916] LustreError: 172177:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6668.448276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.462716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.322836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.330514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.455638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6673.415355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.408840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.333576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.065850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.920300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.113992] LustreError: 173981:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7091.123037] LustreError: 172177:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7763.941488] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7763.949074] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7763.954435] Lustre: Skipped 21 previous similar messages [ 7763.957898] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7765.478821] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7765.487429] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7768.553746] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7768.564617] Lustre: Skipped 1 previous similar message [ 7769.899063] Lustre: server umount lustre-OST0000 complete [ 7777.254840] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7777.270790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7780.272607] Lustre: server umount lustre-MDT0000 complete [ 7783.659706] LustreError: 174002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409279 with bad export cookie 1166261229025776722 [ 7783.669894] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7783.672567] LustreError: 174002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7783.986820] Lustre: server umount lustre-MDT0001 complete [ 7794.189932] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 7797.168692] Key type lgssc unregistered [ 7797.501425] LNet: 227428:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7797.512024] LNetError: 227428:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7797.533134] LNet: Removed LNI 192.168.202.149@tcp [ 7798.270608] Key type .llcrypt unregistered [ 7798.273167] Key type ._llcrypt unregistered [ 7825.292485] Key type ._llcrypt registered [ 7825.294189] Key type .llcrypt registered [ 7825.441144] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 7826.461129] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7826.571225] alg: No test for adler32 (adler32-zlib) [ 7827.625175] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 7827.916855] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 7829.575172] Key type lgssc registered [ 7830.670903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7837.883547] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7837.921626] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7839.421816] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7844.150757] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 7849.952714] LustreError: 228846:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7849.975139] LustreError: 228846:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 7852.378454] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7852.693128] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7856.959246] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 7864.820763] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 7870.603543] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 7877.824594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7878.303815] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7883.947531] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 7884.474318] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7892.060365] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7900.631573] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7903.192116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7903.213490] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7903.230855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7903.249960] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7903.249970] LustreError: 228866:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7908.329459] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7908.331378] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7908.344127] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7908.359292] Lustre: Skipped 1 previous similar message [ 7908.373650] Lustre: Skipped 1 previous similar message [ 7909.664574] Lustre: lustre-OST0000: Not available for connect from 192.168.202.49@tcp (stopping) [ 7911.440116] Lustre: server umount lustre-OST0000 complete [ 7923.322155] LustreError: 228866:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7947.275159] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7947.699726] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7948.719497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7948.766658] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7948.790169] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7948.808734] Lustre: Skipped 1 previous similar message [ 7954.582328] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 7963.030871] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 7964.907597] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7971.632206] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e2692f4d800.ost_server_uuid 50 [ 7973.567574] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e2692f4d800.ost_server_uuid in IDLE state after 0 sec [ 7978.976238] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7978.987723] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7979.022364] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7983.873260] Lustre: server umount lustre-OST0000 complete [ 7989.728308] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7989.737208] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7989.737609] Lustre: Skipped 2 previous similar messages [ 7989.747380] Lustre: Skipped 3 previous similar messages [ 7994.209354] Lustre: server umount lustre-MDT0000 complete [ 7994.849793] LustreError: 229575:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7994.875125] LustreError: 229575:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 7997.896365] LustreError: 228829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409493 with bad export cookie 4868273747810124702 [ 7997.902669] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7997.909764] LustreError: 228829:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7998.214352] Lustre: server umount lustre-MDT0001 complete [ 8012.668889] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 8016.039698] Key type lgssc unregistered [ 8016.401633] LNet: 234053:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8016.406954] LNetError: 234053:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8017.445244] LNet: Removed LNI 192.168.202.149@tcp [ 8018.465949] Key type .llcrypt unregistered [ 8018.467959] Key type ._llcrypt unregistered [ 8052.913840] Key type ._llcrypt registered [ 8052.916097] Key type .llcrypt registered [ 8053.039105] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8054.137398] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8054.147698] alg: No test for adler32 (adler32-zlib) [ 8055.182526] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 8055.427363] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 8057.119199] Key type lgssc registered [ 8058.068958] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8066.981933] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8067.003771] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8068.395399] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8072.856618] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8078.831448] LustreError: 235782:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8078.860451] LustreError: 235782:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 8083.492937] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8083.881723] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8089.391028] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8093.139804] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8100.731976] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8101.033445] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8107.832763] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8113.347135] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8117.409370] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8117.596656] Lustre: lustre-OST0001: new disk, initializing [ 8117.599504] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8117.605307] Lustre: 238089:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8117.718638] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8124.625946] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8126.131173] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8126.139972] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8126.303253] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8139.034890] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8152.579286] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8177.651687] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8177.670981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8177.696567] Lustre: Skipped 3 previous similar messages [ 8177.731420] Lustre: Skipped 3 previous similar messages [ 8182.753408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8182.762957] Lustre: Skipped 1 previous similar message [ 8183.325581] Lustre: server umount lustre-MDT0000 complete [ 8187.875960] LustreError: 235778:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8187.897165] LustreError: 235778:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8188.967463] LustreError: 238088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409684 with bad export cookie 9167455206497316515 [ 8188.971934] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8188.980622] LustreError: 238088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8189.408679] Lustre: server umount lustre-MDT0001 complete [ 8204.193565] Lustre: server umount lustre-OST0000 complete [ 8208.357034] Lustre: 235047:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786409688/real 1786409688] req@ffff92596cec4a80 x1873186181452288/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786409704 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8208.402476] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8209.375394] Lustre: 235046:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786409689/real 1786409689] req@ffff92594814f800 x1873186181452544/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786409705 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8210.152948] Lustre: server umount lustre-OST0001 complete [ 8234.230684] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8246.870985] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8247.152426] LustreError: 241201:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8247.238435] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8251.574413] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8252.392104] LustreError: 241202:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8257.504819] LustreError: 241201:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8261.912044] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8262.468250] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8268.398860] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8272.284417] Lustre: 242344:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8279.559408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8279.846989] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8286.031394] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8288.047569] LustreError: 242699:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8292.158242] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 8295.119196] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8300.540427] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 8301.830242] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8311.114734] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8315.459761] Lustre: 244213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8329.184162] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8329.193231] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8329.225440] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8331.239129] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8331.249185] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8331.253832] Lustre: Skipped 1 previous similar message [ 8331.271869] Lustre: Skipped 3 previous similar messages [ 8333.946228] Lustre: server umount mylustre-MDT0000 complete [ 8336.361671] LustreError: 243167:0:(ldlm_lib.c:1179:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8336.377626] LustreError: 243167:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [ 8337.941081] LustreError: 241178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409833 with bad export cookie 9167455206497319287 [ 8337.948401] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8337.951871] LustreError: 241178:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8338.367312] Lustre: server umount mylustre-MDT0001 complete [ 8353.946212] Lustre: server umount mylustre-OST0000 complete [ 8357.855213] Lustre: 235047:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786409837/real 1786409837] req@ffff925948159880 x1873186181513728/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786409853 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8357.898266] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8357.923685] Lustre: Skipped 1 previous similar message [ 8358.145914] Lustre: server umount mylustre-OST0001 complete [ 8384.049569] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8395.734191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8396.142789] LustreError: 246522:0:(ldlm_lib.c:1179:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8396.247598] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8396.253691] Lustre: Skipped 1 previous similar message [ 8400.273248] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8408.413123] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8413.692406] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8416.489325] Lustre: 247663:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8423.086562] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8430.197350] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8438.277613] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8441.542427] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8441.543771] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49792) [ 8444.910876] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8453.627647] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8457.646682] Lustre: 249537:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8467.425703] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8467.430462] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8467.437420] Lustre: Skipped 2 previous similar messages [ 8467.454224] Lustre: Skipped 3 previous similar messages [ 8467.921243] Lustre: server umount tfs-MDT0000 complete [ 8471.688220] LustreError: 246500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786409967 with bad export cookie 9167455206497321177 [ 8471.697223] LustreError: 246500:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8471.704824] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8472.019178] Lustre: server umount tfs-MDT0001 complete [ 8487.012169] Lustre: server umount tfs-OST0000 complete [ 8488.927165] Lustre: 235044:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786409968/real 1786409968] req@ffff925977c5d500 x1873186181562624/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786409984 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8489.002994] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8489.034667] Lustre: Skipped 1 previous similar message [ 8491.638429] Lustre: server umount tfs-OST0001 complete [ 8517.705681] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8529.901136] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8530.348439] LustreError: 251788:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8530.381180] LustreError: 251788:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 8530.557944] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8530.567059] Lustre: Skipped 3 previous similar messages [ 8535.899513] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8546.117770] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8551.472987] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8554.654247] Lustre: 252930:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8561.830700] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8568.327658] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8575.687048] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49824) [ 8577.092106] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8582.668247] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8583.478452] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8591.752499] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8596.416866] Lustre: 254812:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8604.235471] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 21:01:37 (1786410097) [ 8621.541935] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8621.554413] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8621.560532] Lustre: Skipped 1 previous similar message [ 8621.575416] Lustre: Skipped 3 previous similar messages [ 8635.360100] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8635.564675] Lustre: server umount lustre-MDT0000 complete [ 8638.984168] LustreError: 251763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410134 with bad export cookie 9167455206497322843 [ 8638.995399] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8638.996425] LustreError: 251763:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8642.019812] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8642.022641] Lustre: Skipped 8 previous similar messages [ 8645.527204] Lustre: server umount lustre-MDT0001 complete [ 8655.595493] Lustre: server umount lustre-OST0000 complete [ 8674.271199] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8674.583481] Lustre: server umount lustre-OST0001 complete [ 8682.262698] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 8694.259272] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8737.776488] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8737.776488] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8737.776488] [ 8737.812806] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8738.183895] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8738.217558] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8738.311668] Lustre: lustre-MDT0000: new disk, initializing [ 8738.398715] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8738.403330] Lustre: Skipped 3 previous similar messages [ 8738.431865] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8742.919534] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8754.931319] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8754.931319] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8754.931319] [ 8754.965987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8755.138570] Lustre: 258002:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8755.200912] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8755.205943] Lustre: Skipped 1 previous similar message [ 8755.331711] Lustre: lustre-MDT0001: new disk, initializing [ 8755.539205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8755.554119] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8760.696882] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8774.646358] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8775.080413] Lustre: lustre-OST0000: new disk, initializing [ 8775.090634] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8775.110123] Lustre: 259586:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8776.916955] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8776.941136] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8777.095160] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8783.816510] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8797.001914] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8797.164731] Lustre: lustre-OST0001: new disk, initializing [ 8797.167411] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8797.171912] Lustre: 260462:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8798.603722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8798.608585] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8798.778848] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8803.923697] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8816.875722] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 21:05:10 (1786410310) [ 8832.668399] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 21:05:25 (1786410325) [ 8848.865227] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8848.873964] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8848.886065] LustreError: Skipped 1 previous similar message [ 8848.887149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8848.887155] Lustre: Skipped 2 previous similar messages [ 8848.909425] Lustre: Skipped 5 previous similar messages [ 8853.582550] Lustre: server umount lustre-OST0000 complete [ 8855.023206] LustreError: 259572:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8855.047020] LustreError: 259572:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 15 previous similar messages [ 8858.101574] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8864.001579] Lustre: server umount lustre-MDT0000 complete [ 8867.810533] LustreError: 257995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410363 with bad export cookie 9167455206497324390 [ 8867.817935] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8867.824242] LustreError: 257995:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8882.143527] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8882.747446] Lustre: server umount lustre-MDT0001 complete [ 8896.908248] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 8909.936879] Lustre: server umount lustre-OST0001 complete [ 8917.856254] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 8934.436525] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 8973.797377] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8974.214426] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8974.246703] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8974.326324] Lustre: lustre-MDT0000: new disk, initializing [ 8974.455591] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8979.403602] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 8992.108793] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8992.276035] Lustre: 265019:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8992.336740] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8992.344402] Lustre: Skipped 1 previous similar message [ 8992.459684] Lustre: lustre-MDT0001: new disk, initializing [ 8992.632492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8992.646276] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8998.247845] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9009.192271] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9016.725704] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9024.791817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9025.067572] Lustre: lustre-OST0000: new disk, initializing [ 9025.071808] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9025.076657] Lustre: 266990:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9025.152326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9025.156257] Lustre: Skipped 5 previous similar messages [ 9027.023565] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9027.038112] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9027.201679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9031.550082] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9041.972387] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9052.651404] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9052.656204] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9052.663308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9052.663319] Lustre: Skipped 10 previous similar messages [ 9052.692790] Lustre: Skipped 5 previous similar messages [ 9054.969622] Lustre: server umount lustre-OST0000 complete [ 9059.297318] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9059.310596] LustreError: Skipped 1 previous similar message [ 9065.495231] Lustre: server umount lustre-MDT0000 complete [ 9070.549806] LustreError: 265012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410566 with bad export cookie 9167455206497325867 [ 9070.556713] LustreError: 265012:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9070.557779] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9070.997575] Lustre: server umount lustre-MDT0001 complete [ 9083.933652] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 9086.337941] Key type lgssc unregistered [ 9086.597865] LNet: 268750:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9086.603059] LNetError: 268750:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9087.653665] LNet: Removed LNI 192.168.202.149@tcp [ 9088.593176] Key type .llcrypt unregistered [ 9088.598951] Key type ._llcrypt unregistered [ 9119.857326] Key type ._llcrypt registered [ 9119.858793] Key type .llcrypt registered [ 9119.944965] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 9121.454832] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9121.493161] alg: No test for adler32 (adler32-zlib) [ 9122.531306] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 9122.734717] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 9124.423733] Key type lgssc registered [ 9125.566816] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9133.491996] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9133.518042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9134.992370] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9138.754612] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9144.289053] LustreError: 270214:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9144.313234] LustreError: 270214:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9146.273134] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9146.634411] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9151.340454] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9160.650271] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9168.346492] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9177.468031] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9177.815791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9179.918270] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9185.541605] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9195.258362] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9196.161531] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9200.610607] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9200.613946] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9200.622882] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9202.656150] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9202.660390] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9202.673923] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9202.695770] Lustre: Skipped 1 previous similar message [ 9204.546804] Lustre: server umount lustre-OST0000 complete [ 9212.898499] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9212.906373] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9212.913262] Lustre: Skipped 1 previous similar message [ 9215.011990] Lustre: server umount lustre-MDT0000 complete [ 9218.052116] LustreError: 270215:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9218.864169] LustreError: 272743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410714 with bad export cookie 3342987499799189013 [ 9218.872828] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9218.888552] LustreError: 272743:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9219.357684] Lustre: server umount lustre-MDT0001 complete [ 9230.401233] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 9233.678513] Key type lgssc unregistered [ 9233.961501] LNet: 273798:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9233.969719] LNetError: 273798:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9233.982116] LNet: Removed LNI 192.168.202.149@tcp [ 9234.836595] Key type .llcrypt unregistered [ 9234.838844] Key type ._llcrypt unregistered [ 9257.446277] Key type ._llcrypt registered [ 9257.452844] Key type .llcrypt registered [ 9257.661404] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 9258.776531] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9258.852334] alg: No test for adler32 (adler32-zlib) [ 9260.177806] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 9260.474555] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 9262.247272] Key type lgssc registered [ 9264.017554] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9275.360890] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9275.388887] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9291.105858] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9296.224178] LustreError: 274993:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.149@tcp: failed processing log, type 4: rc = -110 [ 9322.912203] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9329.436682] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9337.410808] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9339.741487] Lustre: server umount lustre-OST0000 complete [ 9357.701593] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 9360.553445] Key type lgssc unregistered [ 9360.816748] LNet: 276623:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9360.828886] LNetError: 276623:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9360.846337] LNet: Removed LNI 192.168.202.149@tcp [ 9361.553179] Key type .llcrypt unregistered [ 9361.556855] Key type ._llcrypt unregistered [ 9403.481976] Key type ._llcrypt registered [ 9403.484769] Key type .llcrypt registered [ 9403.587688] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 9404.879938] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9404.901733] alg: No test for adler32 (adler32-zlib) [ 9406.010970] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [ 9406.273694] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [ 9407.967348] Key type lgssc registered [ 9408.878485] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9422.060137] loop0: detected capacity change from 0 to 204800000 [ 9422.254393] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9422.263544] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9422.775013] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9423.851525] loop0: detected capacity change from 0 to 204800000 [ 9424.028188] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9424.035945] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9424.310969] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9425.272389] loop0: detected capacity change from 0 to 204800000 [ 9425.467082] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9425.474143] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9425.585621] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9426.685587] loop0: detected capacity change from 0 to 204800000 [ 9426.902410] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9428.179888] loop0: detected capacity change from 0 to 204800000 [ 9428.240489] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9432.706942] loop0: detected capacity change from 0 to 204800000 [ 9432.728768] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9436.644630] loop0: detected capacity change from 0 to 204800000 [ 9436.681844] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9440.663058] loop0: detected capacity change from 0 to 204800000 [ 9440.699363] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9445.263586] loop0: detected capacity change from 0 to 204800000 [ 9445.361643] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9445.416592] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9445.499932] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9445.509617] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9445.518357] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9445.537983] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9445.556284] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9445.562800] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9445.571812] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9445.586433] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9445.603719] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9445.618720] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9445.628716] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9445.644900] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9445.656473] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9445.664479] Lustre: 280038:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9451.700203] Lustre: server umount MGS complete [ 9452.650926] loop0: detected capacity change from 0 to 204800000 [ 9452.735061] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9453.180694] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9453.225518] Lustre: lustre-MDT0000: Aborting client recovery [ 9454.154468] loop1: detected capacity change from 0 to 204800000 [ 9454.308200] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9454.354271] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9454.360524] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9454.367460] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9454.393059] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9454.414490] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9454.430562] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9454.453853] Lustre: 280473:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9454.509740] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9454.862293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9454.933584] Lustre: lustre-MDT0001: Aborting client recovery [ 9455.891239] loop2: detected capacity change from 0 to 204800000 [ 9456.040841] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9456.117921] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9456.127400] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9456.143995] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9456.158557] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9456.171301] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9456.179304] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9456.204610] Lustre: 280551:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9456.340037] Lustre: Found index 0 for lustre-OST0000, updating log [ 9456.403326] Lustre: 280569:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9456.469968] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9456.491191] Lustre: lustre-OST0000: Aborting recovery [ 9457.359607] loop3: detected capacity change from 0 to 204800000 [ 9457.506047] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9457.578941] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9457.606586] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9457.615875] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9457.624724] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9457.633277] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9457.640808] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9457.649749] Lustre: 280634:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9457.665955] Lustre: Found index 1 for lustre-OST0001, updating log [ 9457.691245] Lustre: 280639:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9457.770278] Lustre: lustre-OST0001: Aborting recovery [ 9462.822542] LustreError: 280416:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 9463.807168] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9463.821790] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9463.836759] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9463.869948] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9464.829580] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9464.830936] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9465.701387] LustreError: 280383:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 846us: rc = -2 [ 9466.328906] LustreError: 280384:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 919us: rc = -2 [ 9466.343945] LustreError: 280384:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9480.162650] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9480.163241] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9480.185467] Lustre: Skipped 2 previous similar messages [ 9480.222985] Lustre: Skipped 3 previous similar messages [ 9485.281661] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9485.284609] Lustre: Skipped 3 previous similar messages [ 9485.965312] Lustre: server umount lustre-MDT0000 complete [ 9486.805091] LustreError: 280367:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786410982 with bad export cookie 9850627124886547630 [ 9486.814587] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9490.404428] LustreError: 280380:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9490.409524] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9490.433955] LustreError: 280380:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [ 9490.461101] Lustre: Skipped 1 previous similar message [ 9490.471317] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9491.423971] LustreError: 280381:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9491.434709] LustreError: 280381:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9496.544841] LustreError: 280381:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9496.544880] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9496.559671] Lustre: Skipped 4 previous similar messages [ 9501.151466] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9501.521168] Lustre: server umount lustre-MDT0001 complete [ 9508.738266] Lustre: server umount lustre-OST0000 complete [ 9516.342339] Lustre: server umount lustre-OST0001 complete [ 9526.463936] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 21:16:59 (1786411019) [ 9555.688902] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [ 9567.401686] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [ 9607.701925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9607.990286] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9608.028063] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9608.125350] Lustre: lustre-MDT0000: new disk, initializing [ 9608.190339] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9608.193174] Lustre: Skipped 1 previous similar message [ 9608.203963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9612.781646] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9624.031599] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9624.206911] Lustre: 284727:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9624.296323] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9624.299824] Lustre: Skipped 1 previous similar message [ 9624.378990] Lustre: lustre-MDT0001: new disk, initializing [ 9624.515639] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9624.586018] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9624.597168] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9629.809824] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9640.460843] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9647.058799] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9655.416234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9655.748782] Lustre: lustre-OST0000: new disk, initializing [ 9655.753342] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9655.761590] Lustre: 286694:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9655.891586] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9663.104641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9663.123357] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9663.252631] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9663.292982] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9676.166584] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9683.650900] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9699.215761] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9705.911593] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9717.303308] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9770.987703] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9771.007948] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9771.009038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9771.023795] Lustre: Skipped 2 previous similar messages [ 9772.577830] Lustre: server umount lustre-OST0000 complete [ 9778.149669] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9778.156420] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9781.220826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9781.225091] Lustre: Skipped 3 previous similar messages [ 9782.843543] Lustre: server umount lustre-MDT0000 complete [ 9786.337808] LustreError: 284733:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9786.356839] LustreError: 284733:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9786.610263] LustreError: 287992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411282 with bad export cookie 9850627124886550339 [ 9786.617134] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9786.632515] LustreError: 287992:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9787.073982] Lustre: server umount lustre-MDT0001 complete [ 9796.505097] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9802.011553] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9807.613514] Lustre: server umount MGS complete [ 9819.790109] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9820.224792] LustreError: 290634:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9820.248817] LustreError: 290647:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9820.301185] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9825.766198] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9830.886744] LustreError: 290635:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9830.903651] LustreError: 290635:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [ 9838.300039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9846.422618] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9856.981838] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [ 9866.819696] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [ 9876.087954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9876.760711] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9876.773216] Lustre: Skipped 1 previous similar message [ 9885.312733] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [ 9893.557507] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [ 9919.970103] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9919.973416] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9919.987330] Lustre: Skipped 1 previous similar message [ 9919.991256] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9924.598457] Lustre: server umount lustre-OST0000 complete [ 9930.728799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9930.742256] LustreError: Skipped 1 previous similar message [ 9930.746209] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9930.763982] Lustre: Skipped 1 previous similar message [ 9935.849294] Lustre: server umount lustre-MDT0000 complete [ 9936.360714] LustreError: 290635:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9936.382513] LustreError: 290635:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [ 9939.609385] LustreError: 290616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411435 with bad export cookie 9850627124886552292 [ 9939.614982] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9939.617463] LustreError: 290616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9939.926646] Lustre: server umount lustre-MDT0001 complete [ 9953.024515] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [ 9956.277393] Key type lgssc unregistered [ 9956.783721] LNet: 294582:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9956.811320] LNetError: 294582:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9956.831947] LNet: Removed LNI 192.168.202.149@tcp [ 9957.669019] Key type .llcrypt unregistered [ 9957.672506] Key type ._llcrypt unregistered [10002.005333] Key type ._llcrypt registered [10002.007706] Key type .llcrypt registered [10002.124462] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [10020.766378] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [10022.688703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10022.780268] alg: No test for adler32 (adler32-zlib) [10023.947918] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [10024.310817] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [10026.143195] Key type lgssc registered [10027.980143] Lustre: Echo OBD driver; http://www.lustre.org/ [10065.172513] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10065.197461] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10066.466300] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10066.497358] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10066.586646] Lustre: lustre-MDT0000: new disk, initializing [10066.712760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10066.736619] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10071.516878] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10084.139048] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10084.279893] Lustre: 298547:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10084.326215] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10084.329486] Lustre: Skipped 1 previous similar message [10084.399842] Lustre: lustre-MDT0001: new disk, initializing [10084.499763] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10084.546216] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10084.557683] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10091.266126] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10102.143810] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10108.812380] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10115.211921] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10115.608373] Lustre: lustre-OST0000: new disk, initializing [10115.612792] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10115.628351] Lustre: 300514:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10115.729525] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10124.284298] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10125.425133] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10125.449246] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10125.681401] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10135.276643] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10143.030886] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10155.433491] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10163.112324] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10207.712320] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10207.715474] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10207.724771] LustreError: Skipped 1 previous similar message [10207.728690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10207.755449] Lustre: Skipped 1 previous similar message [10209.683241] Lustre: server umount lustre-OST0000 complete [10217.441090] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10217.448059] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10217.460434] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10217.467092] Lustre: Skipped 1 previous similar message [10220.061157] Lustre: server umount lustre-MDT0000 complete [10224.171222] LustreError: 298539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411720 with bad export cookie 1502897278823862347 [10224.178119] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10224.184453] LustreError: 298539:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10224.495545] Lustre: server umount lustre-MDT0001 complete [10234.436756] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10240.360627] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10246.411111] Lustre: server umount MGS complete [10258.543223] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10258.950108] LustreError: 304428:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10258.968742] LustreError: 304441:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10258.969644] LustreError: 304428:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [10259.034557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10263.426389] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10264.033969] LustreError: 304429:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10264.055182] LustreError: 304429:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10268.135705] LustreError: 304428:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10272.220069] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10272.605079] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10277.361356] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10285.991618] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [10293.234215] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [10301.380849] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10301.715888] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10308.650355] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10317.011600] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [10350.050937] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10350.062967] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10350.076196] Lustre: Skipped 1 previous similar message [10350.084341] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10350.095625] Lustre: Skipped 1 previous similar message [10354.831964] Lustre: server umount lustre-OST0000 complete [10359.777353] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10359.795898] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10359.828790] Lustre: Skipped 1 previous similar message [10359.838479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10359.846054] Lustre: Skipped 2 previous similar messages [10365.660642] Lustre: server umount lustre-MDT0000 complete [10366.433109] LustreError: 304428:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10366.448650] LustreError: 304428:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [10370.050655] LustreError: 307154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786411865 with bad export cookie 1502897278823864188 [10370.052098] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10370.066072] LustreError: 307154:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10370.543167] Lustre: server umount lustre-MDT0001 complete [10384.611177] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [10388.547386] Key type lgssc unregistered [10389.036567] LNet: 308375:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10389.044993] LNetError: 308375:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [10389.073386] LNet: Removed LNI 192.168.202.149@tcp [10390.140941] Key type .llcrypt unregistered [10390.144083] Key type ._llcrypt unregistered [10413.351301] Key type ._llcrypt registered [10413.353238] Key type .llcrypt registered [10413.474552] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [10431.525683] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [10433.750286] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10433.804845] alg: No test for adler32 (adler32-zlib) [10435.073321] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [10435.387676] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [10437.137095] Key type lgssc registered [10439.306451] Lustre: Echo OBD driver; http://www.lustre.org/ [10472.657543] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [10490.082493] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10490.100919] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10491.451818] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10491.526184] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10491.633946] Lustre: lustre-MDT0000: new disk, initializing [10491.727861] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10491.762650] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10496.493454] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [10509.987249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10510.376916] Lustre: lustre-OST0000: new disk, initializing [10510.380787] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10510.386118] Lustre: Skipped 1 previous similar message [10510.397414] Lustre: 312428:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10510.568270] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10519.270573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10519.299033] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10519.368288] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10519.652546] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11425.764230] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11425.786685] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11427.225554] Lustre: server umount lustre-MDT0000 complete [11431.649832] LustreError: 311616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786412927 with bad export cookie 16933570318533552712 [11431.654081] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11431.658887] LustreError: 311616:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11431.856112] Lustre: server umount lustre-OST0000 complete [11460.054143] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [11474.040991] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [11517.510544] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11517.702914] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11517.733230] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11517.828685] Lustre: lustre-MDT0000: new disk, initializing [11517.978544] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11518.007655] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11523.891247] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11535.734154] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11535.906730] Lustre: 316773:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11536.029177] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11536.037926] Lustre: Skipped 1 previous similar message [11536.169776] Lustre: lustre-MDT0001: new disk, initializing [11536.287655] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11536.345985] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11536.366249] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11541.821595] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11552.470840] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11559.092382] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11567.442266] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11567.789946] Lustre: lustre-OST0000: new disk, initializing [11567.793198] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11567.804354] Lustre: 318746:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11567.919120] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11575.012642] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11578.008125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11578.018729] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11578.208271] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11585.586670] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11590.638220] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11590.991432] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11596.237325] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11596.538466] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11603.954402] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11603.957545] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11603.972546] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11604.028127] Lustre: Skipped 1 previous similar message [11605.504101] Lustre: server umount lustre-OST0000 complete [11613.152988] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11613.156425] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11613.189330] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11613.191966] Lustre: Skipped 1 previous similar message [11616.119143] Lustre: server umount lustre-MDT0000 complete [11621.249515] LustreError: 316768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413117 with bad export cookie 16933570318536493538 [11621.251622] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11621.270462] LustreError: 316768:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11621.845258] Lustre: server umount lustre-MDT0001 complete [11627.274662] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11629.644995] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 21:52:02 (1786413122) [11638.025911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11638.307694] LustreError: 321060:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11638.317707] LustreError: 321060:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [11638.370532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11643.377926] LustreError: 321061:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11643.911407] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11648.492898] LustreError: 321060:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11653.608629] LustreError: 321061:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11657.158649] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11657.646381] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11664.591376] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11675.021545] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11683.247597] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [11693.000034] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11693.999985] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11701.023948] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11710.022336] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11719.437911] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11719.576591] Lustre: lustre-OST0001: new disk, initializing [11719.579594] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11719.583894] Lustre: 323901:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11719.691255] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11728.053676] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11728.068722] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11728.164785] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11728.883968] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11729.204723] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11729.251682] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11730.280423] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11730.301992] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11732.330191] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11732.340217] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11736.485508] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11736.494304] LustreError: 324206:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11743.933685] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11744.283346] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11744.743938] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11744.754312] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11753.011117] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8e2692f4e000.ost_server_uuid 50 [11754.656740] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8e2692f4e000.ost_server_uuid in FULL state after 0 sec [11760.156855] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8e2692f4e000.ost_server_uuid 50 [11761.185832] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11761.202592] LustreError: 324218:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [11762.325456] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8e2692f4e000.ost_server_uuid in IDLE state after 0 sec [11769.841698] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11778.311452] Lustre: lustre-OST0001: Not available for connect from 192.168.202.49@tcp (stopping) [11778.323903] Lustre: Skipped 2 previous similar messages [11779.567952] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11779.584765] Lustre: Skipped 1 previous similar message [11783.422043] Lustre: lustre-OST0001: Not available for connect from 192.168.202.49@tcp (stopping) [11783.434654] Lustre: Skipped 2 previous similar messages [11783.914352] Lustre: server umount lustre-OST0001 complete [11784.680982] LustreError: 322882:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11784.704584] LustreError: 322882:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [11794.400286] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11794.411208] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11794.418505] Lustre: Skipped 1 previous similar message [11794.423312] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11794.930407] LustreError: 323653:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11794.952261] LustreError: 323653:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [11806.176055] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11806.393682] Lustre: server umount lustre-OST0000 complete [11815.396981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11815.422113] Lustre: Skipped 1 previous similar message [11815.431157] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11815.449758] Lustre: Skipped 6 previous similar messages [11826.143165] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11826.754677] Lustre: server umount lustre-MDT0000 complete [11830.756233] LustreError: 321789:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11830.783169] LustreError: 321789:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 5 previous similar messages [11832.064117] LustreError: 321040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413327 with bad export cookie 16933570318536494420 [11832.100267] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11832.102035] LustreError: 321040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11832.758358] Lustre: server umount lustre-MDT0001 complete [11847.217816] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [11850.742527] Key type lgssc unregistered [11851.182718] LNet: 326594:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11851.187755] LNetError: 326594:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [11851.207046] LNet: Removed LNI 192.168.202.149@tcp [11852.665375] Key type .llcrypt unregistered [11852.667830] Key type ._llcrypt unregistered [11882.079336] Key type ._llcrypt registered [11882.080967] Key type .llcrypt registered [11882.168364] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [11883.372694] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11883.456122] alg: No test for adler32 (adler32-zlib) [11884.636626] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [11884.886205] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [11886.575167] Key type lgssc registered [11887.973818] Lustre: Echo OBD driver; http://www.lustre.org/ [11896.747827] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11896.780271] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11898.303223] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11904.933680] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11908.580345] LustreError: 328018:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11908.598661] LustreError: 328018:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [11913.701137] LustreError: 328017:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11914.770414] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [11918.820973] LustreError: 328018:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11923.025367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11923.296811] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11928.338945] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11939.920193] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11940.238706] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11944.363752] LustreError: 329698:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11944.403338] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11947.578653] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11948.461573] LustreError: 329700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11955.333702] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [11958.763730] LustreError: 329700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11958.801045] LustreError: 329700:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 4 previous similar messages [11963.601448] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11963.812660] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11969.055619] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11972.552226] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [11982.923687] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid [11991.753377] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11992.052404] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12096.998414] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12097.023612] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12097.028087] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12097.041687] Lustre: Skipped 1 previous similar message [12102.119181] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12102.137000] Lustre: Skipped 1 previous similar message [12102.257153] Lustre: server umount lustre-OST0001 complete [12107.262641] LustreError: 329700:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12107.274212] LustreError: 329700:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12108.256950] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12108.264315] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12108.280107] Lustre: Skipped 1 previous similar message [12108.290196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12112.371011] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12112.386227] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12112.399181] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12117.472648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12117.480561] Lustre: Skipped 2 previous similar messages [12122.079848] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12122.328477] Lustre: server umount lustre-OST0000 complete [12127.351736] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12127.359897] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12127.375242] Lustre: Skipped 1 previous similar message [12127.394939] Lustre: Skipped 1 previous similar message [12133.901545] Lustre: server umount lustre-MDT0000 complete [12139.503822] LustreError: 331703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413635 with bad export cookie 2040371630351445455 [12139.507294] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12139.520159] LustreError: 331703:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12140.194943] Lustre: server umount lustre-MDT0001 complete [12157.097683] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [12160.005634] Key type lgssc unregistered [12160.408204] LNet: 332981:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12160.423419] LNetError: 332981:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [12160.483923] LNet: Removed LNI 192.168.202.149@tcp [12161.920134] Key type .llcrypt unregistered [12161.921638] Key type ._llcrypt unregistered [12213.977383] Key type ._llcrypt registered [12213.979594] Key type .llcrypt registered [12214.054249] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [12215.442228] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12215.515339] alg: No test for adler32 (adler32-zlib) [12216.699162] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [12216.979375] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [12218.671271] Key type lgssc registered [12219.922174] Lustre: Echo OBD driver; http://www.lustre.org/ [12232.409950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12232.426816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12234.145967] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12240.564950] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12243.427252] LustreError: 335283:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12243.444675] LustreError: 335283:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [12248.554200] LustreError: 335284:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12249.226917] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12249.581599] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12253.123822] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12255.665824] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12263.103660] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12263.631648] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12270.990310] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12274.916489] LustreError: 337312:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12279.033483] LustreError: 336776:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12279.082876] LustreError: 336776:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12279.102530] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12282.208815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12282.329500] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12287.502429] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12290.406497] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12301.060560] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12309.398827] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12326.369638] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12326.387525] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12326.431238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12328.418755] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12328.422223] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12328.430208] Lustre: Skipped 2 previous similar messages [12328.433884] Lustre: Skipped 2 previous similar messages [12330.526728] Lustre: server umount lustre-MDT0000 complete [12333.555057] LustreError: 336013:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12333.574563] LustreError: 336013:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12334.928501] LustreError: 336023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413830 with bad export cookie 3744040812068316064 [12334.936118] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12334.942535] LustreError: 336023:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12335.627740] Lustre: server umount lustre-MDT0001 complete [12349.788255] Lustre: server umount lustre-OST0000 complete [12364.094816] Lustre: server umount lustre-OST0001 complete [12382.169446] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [12394.505634] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12394.938234] LustreError: 340553:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12395.033043] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12400.049332] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12408.307436] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12408.722321] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12413.974331] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12417.965992] Lustre: 341694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12424.830640] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12425.425246] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12432.084887] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12433.655942] LustreError: 342050:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12433.683163] LustreError: 342050:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 3 previous similar messages [12440.054114] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12441.408263] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12446.800186] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12451.450831] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12462.572707] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12466.129512] Lustre: 343567:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12472.290398] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12472.295033] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12472.318947] Lustre: Skipped 2 previous similar messages [12472.326521] Lustre: Skipped 2 previous similar messages [12477.428506] LustreError: 340548:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12477.447643] LustreError: 340548:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [12477.561408] Lustre: server umount lustre-MDT0000 complete [12481.424944] LustreError: 340534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786413977 with bad export cookie 3744040812068317765 [12481.432619] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12481.452327] LustreError: 340534:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12481.812556] Lustre: server umount lustre-MDT0001 complete [12495.338534] Lustre: server umount lustre-OST0000 complete [12498.339088] Lustre: 334545:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786413978/real 1786413978] req@ffff925847ddb800 x1873190544596352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786413994 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12498.394480] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12498.412197] Lustre: Skipped 1 previous similar message [12499.764370] Lustre: server umount lustre-OST0001 complete [12503.678620] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12505.662817] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12507.339534] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 22:06:40 (1786414000) [12530.226542] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [12538.030089] loop0: detected capacity change from 0 to 1099511627776 [12538.537319] SGI XFS with ACLs, security attributes, quota, no debug enabled [12538.548067] *** VALIDATE xfs *** [12538.559653] XFS (loop0): Mounting V5 Filesystem [12538.613720] XFS (loop0): Ending clean mount [12539.629272] loop1: detected capacity change from 0 to 18691697672192 [12671.587912] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [12684.580162] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [12724.450001] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12724.646561] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12724.670888] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12724.809952] Lustre: lustre-MDT0000: new disk, initializing [12724.922604] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12724.925682] Lustre: Skipped 1 previous similar message [12724.957238] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12730.523430] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12747.766675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12748.080632] Lustre: 348709:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12748.191587] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12748.204792] Lustre: Skipped 1 previous similar message [12748.540701] Lustre: lustre-MDT0001: new disk, initializing [12748.893404] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12748.921770] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12757.972467] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12773.074311] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12782.220444] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12790.417721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12790.673441] Lustre: lustre-OST0000: new disk, initializing [12790.676790] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12790.682994] Lustre: 350678:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12790.857606] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12790.868408] Lustre: Skipped 1 previous similar message [12791.901440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12791.912144] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12792.084535] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12798.752409] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12809.240464] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12815.460873] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12815.750478] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12821.278423] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12821.553288] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12828.134861] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12828.138049] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12828.147338] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12828.147346] Lustre: Skipped 2 previous similar messages [12828.172569] Lustre: Skipped 1 previous similar message [12833.265625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12833.289694] Lustre: Skipped 2 previous similar messages [12838.367628] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12838.602228] Lustre: server umount lustre-OST0000 complete [12843.489304] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12843.504253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12843.512650] Lustre: Skipped 1 previous similar message [12843.532501] Lustre: Skipped 3 previous similar messages [12857.311136] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12857.680534] Lustre: server umount lustre-MDT0000 complete [12858.854928] LustreError: 348721:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12858.884902] LustreError: 348721:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [12862.312557] LustreError: 348701:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786414358 with bad export cookie 3744040812068319319 [12862.326918] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12863.068696] Lustre: server umount lustre-MDT0001 complete [12875.544829] XFS (loop0): Unmounting Filesystem [12881.663902] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 22:12:54 (1786414374) [12890.030759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12897.239952] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12907.681171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12913.202965] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12921.743730] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [12928.009230] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [12934.433097] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12934.680126] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12934.683916] Lustre: Skipped 2 previous similar messages [12941.234244] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [12949.429086] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [12961.264645] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12961.279350] LustreError: Skipped 1 previous similar message [12961.291348] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12961.320767] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12961.323400] Lustre: Skipped 4 previous similar messages [12964.338050] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12967.118783] Lustre: server umount lustre-OST0000 complete [12977.407542] Lustre: server umount lustre-MDT0000 complete [12982.871976] LustreError: 353223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786414478 with bad export cookie 3744040812068320201 [12982.876739] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12982.879645] LustreError: 353223:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12983.535651] Lustre: server umount lustre-MDT0001 complete [12999.001889] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [13002.208309] Key type lgssc unregistered [13002.435404] LNet: 356875:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13002.439466] LNetError: 356875:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [13002.451357] LNet: Removed LNI 192.168.202.149@tcp [13003.498159] Key type .llcrypt unregistered [13003.505650] Key type ._llcrypt unregistered [13042.707526] Key type ._llcrypt registered [13042.719045] Key type .llcrypt registered [13042.955622] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [13043.922703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13043.976168] alg: No test for adler32 (adler32-zlib) [13045.330554] Lustre: Lustre: Build Version: 2.17.54_108_g0ecde8c [13045.597771] LNet: Added LNI 192.168.202.149@tcp [8/256/0/180] [13047.479697] Key type lgssc registered [13049.130613] Lustre: Echo OBD driver; http://www.lustre.org/ [13060.005796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13060.018845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13061.398257] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13067.928282] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13071.843466] LustreError: 358444:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13071.909173] LustreError: 358444:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13076.971298] LustreError: 358445:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13079.765288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13080.314452] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13086.643261] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13096.812806] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [13105.068895] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [13115.053305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13115.525818] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13125.791555] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13136.533575] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [13147.104749] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13147.118840] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13147.156381] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13147.639252] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13147.663952] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13148.932123] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.49@tcp (stopping) [13152.739542] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13152.757376] Lustre: Skipped 1 previous similar message [13157.860713] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13157.874996] Lustre: Skipped 2 previous similar messages [13159.391144] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13159.870263] Lustre: server umount lustre-MDT0001 complete [13162.988222] LustreError: 358444:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13163.014331] LustreError: 358444:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 1 previous similar message [13168.111928] LustreError: 358441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13168.138304] LustreError: 358441:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13173.232710] LustreError: 358441:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13173.286659] LustreError: 358441:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [13173.548823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13173.717870] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13173.724280] Lustre: Found index 1 for lustre-MDT0001, updating log [13173.789787] Lustre: 358434:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13174.399935] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13179.907229] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13181.796950] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13183.608822] LustreError: 361954:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13183.618645] LustreError: 361954:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13183.640102] LustreError: 358436:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13183.799973] LustreError: 361961:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.149@tcp: cfg command failed: rc = -17 [13183.814711] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13183.814711] [13198.340176] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13198.395614] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13198.397706] Lustre: Skipped 1 previous similar message [13204.214477] Lustre: server umount lustre-MDT0001 complete [13205.252561] LustreError: 358440:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13216.970993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13217.154760] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13217.162303] Lustre: Found index 1 for lustre-MDT0001, updating log [13217.238331] Lustre: 358433:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13217.601617] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13222.910055] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13223.281395] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13227.509456] LustreError: 363127:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13227.524542] LustreError: 363127:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13227.537080] LustreError: 358436:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13227.540294] LustreError: 358436:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13227.609670] LustreError: 363130:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13227.618298] LustreError: 363130:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13227.623214] LustreError: 363130:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.149@tcp: cfg command failed: rc = -17 [13227.628698] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.149@tcp [13227.628698] [13530.093385] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13530.096925] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13530.146720] Lustre: Skipped 1 previous similar message [13530.148836] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13530.148843] Lustre: Skipped 4 previous similar messages [13535.832225] Lustre: server umount lustre-MDT0001 complete [13538.035352] LustreError: 362938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.49@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13538.056681] LustreError: 362938:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 6 previous similar messages [13548.926316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13549.128634] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13549.140244] Lustre: Found index 1 for lustre-MDT0001, updating log [13549.182874] Lustre: 358431:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13549.641413] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13554.705603] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13555.441512] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13558.779996] LustreError: 371025:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13558.789288] LustreError: 371025:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13558.800843] LustreError: 358436:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13558.814614] LustreError: 358436:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13558.894909] LustreError: 371032:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13558.912908] LustreError: 371032:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.202.149@tcp: cfg command failed: rc = -114 [13558.919374] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13558.919374] [13882.336444] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13882.350202] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13882.355519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13882.355528] Lustre: Skipped 4 previous similar messages [13882.410899] Lustre: Skipped 3 previous similar messages [13887.146546] Lustre: server umount lustre-MDT0000 complete [13887.466655] LustreError: 362938:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13887.494241] LustreError: 362938:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 8 previous similar messages [13892.159792] LustreError: 358424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786415388 with bad export cookie 12347685979108183856 [13892.162712] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13892.170254] LustreError: 358424:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13892.585032] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13898.801124] Lustre: server umount lustre-MDT0001 complete [13903.466097] Lustre: server umount lustre-OST0000 complete [13916.106981] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [13926.779606] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [13968.446651] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 22:31:01 (1786415461) [13976.036973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13976.359744] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13976.414735] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13976.507428] Lustre: lustre-MDT0000: new disk, initializing [13976.672353] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13976.701571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13982.434307] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [13994.720863] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13994.861570] Lustre: 381181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13994.936354] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13994.940069] Lustre: Skipped 1 previous similar message [13995.023931] Lustre: lustre-MDT0001: new disk, initializing [13995.115601] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13995.149707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13995.161246] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14000.723731] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [14011.907661] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid [14018.473721] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid [14027.741948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14028.028720] Lustre: lustre-OST0000: new disk, initializing [14028.032855] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14028.038387] Lustre: 383152:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14028.138261] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14034.394535] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_default_debug -1 all [14034.512944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14034.529949] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14034.827325] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14046.263767] Lustre: DEBUG MARKER: oleg249-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid [14055.989823] Lustre: server umount lustre-MDT0000 complete [14056.432320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14056.443465] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14056.456992] LustreError: 381193:0:(ldlm_lib.c:1179:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14056.468734] LustreError: 381193:0:(ldlm_lib.c:1179:target_handle_connect()) Skipped 2 previous similar messages [14060.779536] LustreError: 381978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786415556 with bad export cookie 12347685979108185816 [14060.789850] LustreError: MGC192.168.202.149@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14060.794712] LustreError: 381978:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [14060.795132] LustreError: 381990:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14060.795280] LustreError: 381990:0:(client.c:1381:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000c94f3a42 x1873191414059520/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [14061.203664] Lustre: server umount lustre-MDT0001 complete [14075.539418] Lustre: server umount lustre-OST0000 complete [14110.889452] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing set_hostid [14125.999453] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing load_modules_local [14163.298636] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13877 sec ========= 22:34:16 (1786415656) [14165.658631] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 22:34:18 (1786415658) === [14169.771442] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 22:34:22 (1786415662) === [14214.594644] Lustre: DEBUG MARKER: oleg249-server.virtnet: executing unload_modules_local [14217.741712] Key type lgssc unregistered [14218.165840] LNet: 389544:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14218.174894] LNetError: 389544:0:(acceptor.c:246:lnet_acceptor_remove_socket()) Interface ens2 not found [14218.218491] LNet: Removed LNI 192.168.202.149@tcp [14219.199471] Key type .llcrypt unregistered [14219.205831] Key type ._llcrypt unregistered