[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 550009746 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.003143] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.007963] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009016] pid_max: default: 32768 minimum: 301 [ 0.010142] LSM: Security Framework initializing [ 0.011069] Yama: becoming mindful. [ 0.012055] SELinux: Initializing. [ 0.013075] *** VALIDATE selinux *** [ 0.022136] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026732] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028073] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029093] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030129] *** VALIDATE tmpfs *** [ 0.032079] *** VALIDATE proc *** [ 0.033074] *** VALIDATE cgroup *** [ 0.033840] *** VALIDATE cgroup2 *** [ 0.034238] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035125] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.036008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.037028] Spectre V2 : User space: Vulnerable [ 0.038006] Speculative Store Bypass: Vulnerable [ 0.041539] debug: unmapping init [mem 0xffffffff98e59000-0xffffffff98e60fff] [ 0.044224] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045819] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046033] ... version: 2 [ 0.047015] ... bit width: 48 [ 0.048012] ... generic registers: 4 [ 0.049019] ... value mask: 0000ffffffffffff [ 0.050016] ... max period: 00007fffffffffff [ 0.051024] ... fixed-purpose events: 3 [ 0.052018] ... event mask: 000000070000000f [ 0.053351] rcu: Hierarchical SRCU implementation. [ 0.055523] smp: Bringing up secondary CPUs ... [ 0.056621] x86: Booting SMP configuration: [ 0.057039] .... node #0, CPUs: #1 #2 #3 [ 0.061103] smp: Brought up 1 node, 4 CPUs [ 0.063022] smpboot: Max logical packages: 1 [ 0.064022] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.141173] node 0 deferred pages initialised in 73ms [ 0.147035] devtmpfs: initialized [ 0.148212] x86/mm: Memory block size: 128MB [ 0.151000] gcov: version magic: 0x41383552 [ 0.153306] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.158120] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.161399] pinctrl core: initialized pinctrl subsystem [ 0.164360] [ 0.165026] ************************************************************* [ 0.169019] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.172018] ** ** [ 0.175023] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.178025] ** ** [ 0.181016] ** This means that this kernel is built to expose internal ** [ 0.184016] ** IOMMU data structures, which may compromise security on ** [ 0.187016] ** your system. ** [ 0.190020] ** ** [ 0.193015] ** If you see this message and you are not debugging the ** [ 0.196017] ** kernel, report this immediately to your vendor! ** [ 0.199025] ** ** [ 0.202013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.204020] ************************************************************* [ 0.207617] NET: Registered protocol family 16 [ 0.209326] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.211064] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.213049] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.216039] cpuidle: using governor menu [ 0.218788] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.219701] PCI: Using configuration type 1 for base access [ 0.221149] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.234078] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.236019] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.240102] cryptd: max_cpu_qlen set to 1000 [ 0.244332] ACPI: Added _OSI(Module Device) [ 0.246023] ACPI: Added _OSI(Processor Device) [ 0.247011] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.248012] ACPI: Added _OSI(Processor Aggregator Device) [ 0.253082] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.260391] ACPI: Interpreter enabled [ 0.262059] ACPI: PM: (supports S0 S3 S4 S5) [ 0.263024] ACPI: Using IOAPIC for interrupt routing [ 0.265167] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.270161] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.281558] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.286058] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.288035] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.290080] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.293372] acpiphp: Slot [2] registered [ 0.295186] acpiphp: Slot [5] registered [ 0.296132] acpiphp: Slot [6] registered [ 0.297183] acpiphp: Slot [7] registered [ 0.298201] acpiphp: Slot [8] registered [ 0.299140] acpiphp: Slot [9] registered [ 0.301283] acpiphp: Slot [10] registered [ 0.302175] acpiphp: Slot [3] registered [ 0.303088] acpiphp: Slot [4] registered [ 0.305253] acpiphp: Slot [11] registered [ 0.307128] acpiphp: Slot [12] registered [ 0.309214] acpiphp: Slot [13] registered [ 0.311136] acpiphp: Slot [14] registered [ 0.314165] acpiphp: Slot [15] registered [ 0.315198] acpiphp: Slot [16] registered [ 0.318270] acpiphp: Slot [17] registered [ 0.320151] acpiphp: Slot [18] registered [ 0.322179] acpiphp: Slot [19] registered [ 0.325162] acpiphp: Slot [20] registered [ 0.327199] acpiphp: Slot [21] registered [ 0.329189] acpiphp: Slot [22] registered [ 0.332208] acpiphp: Slot [23] registered [ 0.334129] acpiphp: Slot [24] registered [ 0.336113] acpiphp: Slot [25] registered [ 0.338170] acpiphp: Slot [26] registered [ 0.339185] acpiphp: Slot [27] registered [ 0.340108] acpiphp: Slot [28] registered [ 0.341099] acpiphp: Slot [29] registered [ 0.342108] acpiphp: Slot [30] registered [ 0.344110] acpiphp: Slot [31] registered [ 0.345074] PCI host bridge to bus 0000:00 [ 0.346027] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.348036] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.351048] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.353032] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.355049] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.358053] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.361297] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.364200] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.366305] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.376037] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.381062] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.383031] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.386030] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.388023] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.391476] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.393539] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.395029] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.396610] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.401012] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.417815] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.421022] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.425517] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.438025] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.449023] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.471026] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.490618] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.503016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.514026] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.532030] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.542432] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.552020] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.564024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.598026] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.613902] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.630020] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.636018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.649017] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.659075] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.666023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.678019] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.695026] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.703194] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.709027] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.715017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.728028] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.738251] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.740424] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.743491] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.745430] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.748260] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.753153] iommu: Default domain type: Passthrough [ 0.755607] SCSI subsystem initialized [ 0.757199] ACPI: bus type USB registered [ 0.759146] usbcore: registered new interface driver usbfs [ 0.761121] usbcore: registered new interface driver hub [ 0.763204] usbcore: registered new device driver usb [ 0.765346] pps_core: LinuxPPS API ver. 1 registered [ 0.767016] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.770102] PTP clock support registered [ 0.773133] EDAC MC: Ver: 3.0.0 [ 0.775183] PCI: Using ACPI for IRQ routing [ 0.777637] NetLabel: Initializing [ 0.778016] NetLabel: domain hash size = 128 [ 0.780014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.782108] NetLabel: unlabeled traffic allowed by default [ 0.785124] vgaarb: loaded [ 0.786281] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.788009] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.792021] clocksource: Switched to clocksource kvm-clock [ 0.911257] VFS: Disk quotas dquot_6.6.0 [ 0.912798] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.915324] *** VALIDATE ramfs *** [ 0.916135] *** VALIDATE hugetlbfs *** [ 0.917174] pnp: PnP ACPI init [ 0.919536] pnp: PnP ACPI: found 6 devices [ 0.937851] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.940598] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.942101] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.943464] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.945203] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.946809] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.949520] NET: Registered protocol family 2 [ 0.951934] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.955388] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.957803] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.964184] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.969109] TCP: Hash tables configured (established 65536 bind 65536) [ 0.973079] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.976709] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.980487] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.983977] NET: Registered protocol family 1 [ 0.987844] RPC: Registered named UNIX socket transport module. [ 0.989519] RPC: Registered udp transport module. [ 0.991015] RPC: Registered tcp transport module. [ 0.992110] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.994038] NET: Registered protocol family 44 [ 0.995167] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.996477] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.997839] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.999298] PCI: CLS 0 bytes, default 64 [ 1.000455] Unpacking initramfs... [ 2.347415] debug: unmapping init [mem 0xffff8e0bbcc54000-0xffff8e0bbffbffff] [ 2.350190] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.351755] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.353397] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.860248] Initialise system trusted keyrings [ 2.862021] Key type blacklist registered [ 2.864042] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.875811] zbud: loaded [ 2.880262] *** VALIDATE nfs *** [ 2.881898] *** VALIDATE nfs4 *** [ 2.883838] pstore: using deflate compression [ 2.888554] Platform Keyring initialized [ 2.973863] NET: Registered protocol family 38 [ 2.975025] Key type asymmetric registered [ 2.976030] Asymmetric key parser 'x509' registered [ 2.977534] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 2.980124] io scheduler mq-deadline registered [ 2.981325] io scheduler kyber registered [ 2.982446] io scheduler bfq registered [ 2.984040] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 2.986561] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 2.988793] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 2.991243] ACPI: Power Button [PWRF] [ 2.996109] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.001986] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.011706] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.017283] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.027849] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.053759] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.081581] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.084993] Non-volatile memory driver v1.3 [ 3.085907] Linux agpgart interface v0.103 [ 3.112262] virtio_blk virtio1: [vda] 149760 512-byte logical blocks (76.7 MB/73.1 MiB) [ 3.114319] vda: detected capacity change from 0 to 76677120 [ 3.126753] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.128821] vdb: detected capacity change from 0 to 1073741824 [ 3.141718] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.143694] vdc: detected capacity change from 0 to 2621440000 [ 3.156334] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.158393] vdd: detected capacity change from 0 to 2621440000 [ 3.170112] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.172041] vde: detected capacity change from 0 to 4294967296 [ 3.183950] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.186721] vdf: detected capacity change from 0 to 4294967296 [ 3.193566] libphy: Fixed MDIO Bus: probed [ 3.204515] usbcore: registered new interface driver usbserial_generic [ 3.206511] usbserial: USB Serial support registered for generic [ 3.208376] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.211986] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.213833] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.216414] mousedev: PS/2 mouse device common for all mice [ 3.219901] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.223112] rtc_cmos 00:05: RTC can wake from S4 [ 3.227528] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.227864] rtc_cmos 00:05: registered as rtc0 [ 3.232321] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.236210] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.236232] intel_pstate: CPU model not supported [ 3.242536] hid: raw HID events driver (C) Jiri Kosina [ 3.244427] usbcore: registered new interface driver usbhid [ 3.245848] usbhid: USB HID core driver [ 3.247149] drop_monitor: Initializing network drop monitor service [ 3.248888] Initializing XFRM netlink socket [ 3.250428] NET: Registered protocol family 10 [ 3.252565] Segment Routing with IPv6 [ 3.253575] NET: Registered protocol family 17 [ 3.255034] mpls_gso: MPLS GSO support [ 3.259506] RAS: Correctable Errors collector initialized. [ 3.261653] AVX version of gcm_enc/dec engaged. [ 3.262731] AES CTR mode by8 optimization enabled [ 3.326551] sched_clock: Marking stable (3326529694, 0)->(4243199270, -916669576) [ 3.329311] registered taskstats version 1 [ 3.330798] Loading compiled-in X.509 certificates [ 3.332770] zswap: loaded using pool lzo/zbud [ 3.353696] Key type big_key registered [ 3.365947] Key type encrypted registered [ 3.367587] ima: No TPM chip found, activating TPM-bypass! [ 3.369453] ima: Allocated hash algorithm: sha1 [ 3.370694] ima: No architecture policies found [ 3.372093] evm: Initialising EVM extended attributes: [ 3.373527] evm: security.selinux [ 3.374467] evm: security.ima [ 3.375293] evm: security.capability [ 3.376304] evm: HMAC attrs: 0x1 [ 3.378138] rtc_cmos 00:05: setting system clock to 2026-09-03 12:14:19 UTC (1788437659) [ 3.382875] debug: unmapping init [mem 0xffffffff99e03000-0xffffffff99ffffff] [ 3.385300] debug: unmapping init [mem 0xffffffff98b82000-0xffffffff98e58fff] [ 3.393065] Write protecting the kernel read-only data: 28672k [ 3.395582] debug: unmapping init [mem 0xffffffff97203000-0xffffffff973fffff] [ 3.397646] debug: unmapping init [mem 0xffffffff97b14000-0xffffffff97bfffff] [ 3.427475] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.433560] systemd[1]: Detected virtualization kvm. [ 3.434797] systemd[1]: Detected architecture x86-64. [ 3.436034] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.462677] systemd[1]: No hostname configured. [ 3.463646] systemd[1]: Set hostname to . [ 3.464891] random: systemd: uninitialized urandom read (16 bytes read) [ 3.466499] systemd[1]: Initializing machine ID from random generator. [ 3.504766] random: ln: uninitialized urandom read (6 bytes read) [ 3.592550] random: systemd: uninitialized urandom read (16 bytes read) [ 3.594443] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 3.598280] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.601033] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. Starting Create Volatile Files and Directories... [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... [ OK ] Reached target Timers. [ OK ] Reached target Slices. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.308657] device-mapper: uevent: version 1.0.3 [ 4.310886] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [[ 4.912211] random: fast init done  OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.076474] virtio_net virtio0 ens2: renamed from eth0 [ 5.169277] scsi host0: ata_piix [ 5.186532] scsi host1: ata_piix [ 5.188510] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.191577] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.889182] random: crng init done [ 9.892486] random: 7 urandom warning(s) missed due to ratelimiting [ 10.124297] dracut-initqueue[595]: RTNETLINK answers: File exists [ 10.413237] hrtimer: interrupt took 3152214 ns Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.945537] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Swap. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 15.214666] printk: systemd: 26 output lines suppressed due to ratelimiting [ 16.227885] SELinux: Disabled at runtime. [ 16.401757] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 16.423733] systemd[1]: Detected virtualization kvm. [ 16.426401] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 17.998293] systemd[1]: initrd-switch-root.service: Succeeded. [ 18.007548] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 18.031483] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 18.047397] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 18.065429] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 18.084789] systemd[1]: Starting Journal Service... Starting Journal Service... [ 18.100574] systemd[1]: Listening on initctl Compatibility Named Pipe. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Listening on Process Core Dump Socket. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Mounting POSIX Message Queue File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. Activating swap /dev/disk/by-label/SWAP... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ 18.375813] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Kernel Socket. Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Created slice User and Session Slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Reached target Paths. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... Mounting Huge Pages File System... [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Huge Pages File System. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 19.847818] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 20.989540] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 21.059414] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 22.010750] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 22.089840] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 27.321955] Key type dns_resolver registered [ *** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is runni[ 28.390659] NFS: Registering the id_resolver key type ng for Configur…only root support (10s / no limit)[ 28.395872] Key type id_resolver registered [ 28.400229] Key type id_legacy registered [ ***] A start job is running for Configur…only root support (11s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started D-Bus System Message Bus. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting Restore /run/initramfs on shutdown... Starting Login Service... Starting Network Manager... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting Network Manager Wait Online... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg233-server login: [ 99.834121] libcfs: loading out-of-tree module taints kernel. [ 99.897895] Key type ._llcrypt registered [ 99.912311] Key type .llcrypt registered [ 99.997464] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 119.107412] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 120.590646] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 120.615332] alg: No test for adler32 (adler32-zlib) [ 122.771585] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 124.397513] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 126.368279] Key type lgssc registered [ 129.249313] Lustre: Echo OBD driver; http://www.lustre.org/ [ 147.207048] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 195.542486] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 211.705195] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 211.773407] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 213.136515] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 213.199520] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 213.391331] Lustre: lustre-MDT0000: new disk, initializing [ 213.476609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 213.503965] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 218.065862] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 233.332087] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 233.446855] Lustre: 6519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 233.476563] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 233.481191] Lustre: Skipped 1 previous similar message [ 233.568926] Lustre: lustre-MDT0001: new disk, initializing [ 233.633820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 233.661471] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 233.674762] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 238.259557] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 243.467498] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 252.737399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 252.907249] Lustre: lustre-OST0000: new disk, initializing [ 252.910352] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 252.915673] Lustre: 8456:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 252.965753] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 258.625241] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 258.652589] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 258.787184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 260.176433] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 277.345685] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 277.471561] Lustre: lustre-OST0001: new disk, initializing [ 277.476869] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 277.482113] Lustre: 9529:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 277.549255] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 285.271417] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 285.300075] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 285.371120] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 285.631854] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 297.401946] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 308.224270] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 315.093932] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing check_logdir /tmp/testlogs/ [ 320.641459] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing yml_node [ 325.374425] Lustre: DEBUG MARKER: Client: 2.17.58.2 [ 327.936115] Lustre: DEBUG MARKER: MDS: 2.17.58.2 [ 330.869909] Lustre: DEBUG MARKER: OSS: 2.17.58.2 [ 332.421681] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Sep 3 08:19:48 EDT 2026 [ 350.627685] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 352.147283] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 372.193805] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 372.197429] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.221995] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 372.705567] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 372.731171] Lustre: Skipped 2 previous similar messages [ 372.738720] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 372.742504] Lustre: Skipped 2 previous similar messages [ 376.647833] Lustre: server umount lustre-MDT0000 complete [ 380.653808] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438036 with bad export cookie 12845101497105384315 [ 380.657503] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 380.671195] LustreError: 6510:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 382.945614] LustreError: 10363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 382.948086] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 382.955590] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 382.966265] LustreError: 10363:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 382.981271] Lustre: Skipped 1 previous similar message [ 385.953101] LustreError: 10363:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 385.953244] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 385.984292] Lustre: Skipped 2 previous similar messages [ 387.224644] Lustre: server umount lustre-MDT0001 complete [ 391.559136] Lustre: server umount lustre-OST0000 complete [ 395.850945] Lustre: server umount lustre-OST0001 complete [ 402.743959] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 413.087849] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 456.443411] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 456.917809] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 456.985868] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 457.102560] Lustre: lustre-MDT0000: new disk, initializing [ 457.233439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 457.248100] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 462.115550] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 474.382025] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 474.554328] Lustre: 15085:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 474.617541] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 474.621942] Lustre: Skipped 1 previous similar message [ 474.692832] Lustre: lustre-MDT0001: new disk, initializing [ 474.802522] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 474.857166] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 474.881194] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 480.132480] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 490.204582] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 496.025927] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 502.709419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 503.105510] Lustre: lustre-OST0000: new disk, initializing [ 503.112338] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 503.118155] Lustre: 17054:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 503.195702] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 504.321452] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 504.336548] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 504.525839] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 510.507240] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 520.835466] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 525.809757] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 526.020688] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 530.996221] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 531.222654] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 535.009757] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 535.017427] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.037536] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 539.428301] Lustre: server umount lustre-OST0000 complete [ 545.761397] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.766483] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 545.772994] Lustre: Skipped 2 previous similar messages [ 545.783603] Lustre: Skipped 2 previous similar messages [ 549.543516] Lustre: server umount lustre-MDT0000 complete [ 550.890818] LustreError: 15097:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 550.907284] LustreError: 15097:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 553.271894] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438209 with bad export cookie 12845101497105385848 [ 553.273962] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 553.282967] LustreError: 15077:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 553.811862] Lustre: server umount lustre-MDT0001 complete [ 559.323563] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 08:23:35 (1788438215) [ 566.226668] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 566.681799] LustreError: 19319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 566.755686] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 571.881826] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 571.957547] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 581.092037] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 581.129452] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 581.842532] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 588.133766] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 595.478425] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 601.934920] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 611.886462] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 619.318653] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 627.985521] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 643.352272] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 652.274277] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 652.280072] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 652.303182] Lustre: Skipped 2 previous similar messages [ 652.325458] Lustre: Skipped 2 previous similar messages [ 656.201622] Lustre: server umount lustre-MDT0000 complete [ 657.386847] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 657.397281] LustreError: 19320:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 660.065591] LustreError: 19301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438316 with bad export cookie 12845101497105386730 [ 660.081200] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 660.086863] LustreError: 19301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 660.742921] Lustre: server umount lustre-MDT0001 complete [ 675.811748] Lustre: server umount lustre-OST0000 complete [ 699.386868] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 710.602116] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 710.992880] LustreError: 23725:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 711.091777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 711.107647] Lustre: Skipped 2 previous similar messages [ 716.075343] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 724.242229] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 730.174268] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 733.892361] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 741.653963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 749.116724] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 758.702670] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 758.879329] Lustre: lustre-OST0001: new disk, initializing [ 758.886683] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 758.893964] Lustre: 26041:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 760.392718] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 760.412857] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 760.559045] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 768.199201] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 779.621781] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 782.981551] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 796.130950] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 796.144245] LustreError: Skipped 1 previous similar message [ 796.150840] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 796.164230] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 801.548788] Lustre: server umount lustre-MDT0000 complete [ 801.797447] LustreError: 23721:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 801.811213] LustreError: 23721:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 805.475609] LustreError: 23707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438461 with bad export cookie 12845101497105387577 [ 805.494801] LustreError: 23707:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 805.504212] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 805.808661] Lustre: server umount lustre-MDT0001 complete [ 819.954282] Lustre: server umount lustre-OST0000 complete [ 823.266562] Lustre: 3652:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788438463/real 1788438463] req@ffff8e0b05589c00 x1875312933854592/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788438479 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 824.609949] Lustre: server umount lustre-OST0001 complete [ 844.342783] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 855.864288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 856.383204] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 856.400267] Lustre: Skipped 3 previous similar messages [ 861.626727] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 871.206233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 877.365904] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 880.602777] Lustre: 30206:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 887.050233] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 895.510626] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 906.071178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 913.643550] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 922.968381] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 926.875507] Lustre: 32088:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 933.354675] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 933.361900] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 933.371794] Lustre: Skipped 4 previous similar messages [ 933.382465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 933.391550] Lustre: Skipped 3 previous similar messages [ 939.400167] Lustre: server umount lustre-MDT0000 complete [ 942.566282] LustreError: 29054:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 942.603518] LustreError: 29054:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 943.673978] LustreError: 29038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438599 with bad export cookie 12845101497105388872 [ 943.682980] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 943.688596] LustreError: 29038:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 944.057527] Lustre: server umount lustre-MDT0001 complete [ 959.340391] Lustre: server umount lustre-OST0000 complete [ 963.552527] Lustre: 3652:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788438603/real 1788438603] req@ffff8e0b0417ce00 x1875312933905408/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788438619 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 964.514550] Lustre: server umount lustre-OST0001 complete [ 974.399373] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 08:30:30 (1788438630) [ 1007.500567] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1020.491645] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1027.460500] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1037.007191] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1042.162222] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1045.685278] Lustre: 36040:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1052.598768] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1060.594374] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1070.449066] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1076.913548] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1084.934878] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1088.312751] Lustre: 37918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1096.683111] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1096.687463] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1096.711339] Lustre: Skipped 7 previous similar messages [ 1096.729127] Lustre: Skipped 7 previous similar messages [ 1100.457615] Lustre: server umount lustre-MDT0000 complete [ 1104.218067] LustreError: 34871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438760 with bad export cookie 12845101497105389971 [ 1104.223786] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1104.244160] LustreError: 34871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1104.703703] Lustre: server umount lustre-MDT0001 complete [ 1119.416585] Lustre: server umount lustre-OST0000 complete [ 1123.361475] Lustre: 3652:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788438763/real 1788438763] req@ffff8e0c2ef08700 x1875312933956352/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788438779 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1124.637572] Lustre: server umount lustre-OST0001 complete [ 1137.921747] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 08:33:13 (1788438793) [ 1155.433198] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1167.366320] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1168.012642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1168.017286] Lustre: Skipped 7 previous similar messages [ 1173.615222] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1184.171082] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1191.080968] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1195.018930] Lustre: 41209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1203.164796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1209.779223] LustreError: 41564:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1209.802019] LustreError: 41564:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 15 previous similar messages [ 1211.032707] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1221.819727] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1230.021833] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1238.511514] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1242.733232] Lustre: 43089:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1243.905793] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1243.909541] Lustre: Skipped 1 previous similar message [ 1250.285654] Lustre: server umount lustre-MDT0000 complete [ 1251.298827] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1251.312032] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1251.330570] Lustre: Skipped 4 previous similar messages [ 1255.195806] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438911 with bad export cookie 12845101497105391084 [ 1255.205327] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1255.212365] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1261.763055] Lustre: server umount lustre-MDT0001 complete [ 1272.052224] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1279.973291] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xb242f7c15ece223e [ 1280.006590] Lustre: MGC192.168.202.133@tcp: Connection restored to 0@lo (at 0@lo) [ 1286.336369] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1290.744940] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1296.199415] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1301.433700] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1301.995284] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1302.002201] Lustre: Skipped 1 previous similar message [ 1311.258082] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1312.980233] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1319.469920] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1321.310246] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1332.738377] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1334.422814] Lustre: server umount lustre-MDT0000 complete [ 1339.111315] LustreError: 40039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788438995 with bad export cookie 12845101497105392190 [ 1339.114447] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1339.119225] LustreError: 40039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1339.850736] Lustre: server umount lustre-MDT0001 complete [ 1354.424715] Lustre: server umount lustre-OST0000 complete [ 1368.219765] Lustre: server umount lustre-OST0001 complete [ 1379.453956] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 08:37:15 (1788439035) [ 1398.082705] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1410.248676] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1416.607411] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1426.272638] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1433.145199] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1436.752350] Lustre: 48852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1445.663460] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1454.558816] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1465.792071] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1474.072103] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1482.498201] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1487.803665] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1500.580689] Lustre: server umount lustre-MDT0000 complete [ 1504.683747] LustreError: 50738:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788439160 with bad export cookie 12845101497105393009 [ 1504.690526] LustreError: 50738:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1504.692956] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1504.971227] Lustre: server umount lustre-MDT0001 complete [ 1520.098312] Lustre: server umount lustre-OST0000 complete [ 1523.488934] Lustre: 3653:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788439163/real 1788439163] req@ffff8e0b034b1180 x1875312934110208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788439179 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1523.536277] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1523.556402] Lustre: Skipped 10 previous similar messages [ 1524.573262] Lustre: server umount lustre-OST0001 complete [ 1535.212493] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 08:39:51 (1788439191) [ 1538.513491] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1540.578696] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 08:39:56 (1788439196) [ 1552.051648] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 1561.522995] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1586.299820] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1586.672576] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1586.776270] Lustre: lustre-MDT0000: new disk, initializing [ 1586.889256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1592.226573] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1604.260712] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1613.187809] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1613.498399] Lustre: lustre-OST0000: new disk, initializing [ 1613.503173] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1613.511567] Lustre: Skipped 1 previous similar message [ 1613.521276] Lustre: 54930:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1615.178363] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1615.195136] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1615.228166] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1621.191996] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1632.302114] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1635.219377] Lustre: DEBUG MARKER: create test files [ 1675.525690] Lustre: server umount lustre-OST0000 complete [ 1692.021169] Lustre: server umount lustre-MDT0000 complete [ 1707.182589] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1711.020825] Key type lgssc unregistered [ 1711.339110] LNet: 56567:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1711.348288] LNetError: 56567:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1711.380678] LNet: Removed LNI 192.168.202.133@tcp [ 1712.533746] Key type .llcrypt unregistered [ 1712.540301] Key type ._llcrypt unregistered [ 1745.576927] Key type ._llcrypt registered [ 1745.581109] Key type .llcrypt registered [ 1745.699171] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1747.279660] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1747.296323] alg: No test for adler32 (adler32-zlib) [ 1748.505039] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 1748.864946] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 1750.552974] Key type lgssc registered [ 1752.002802] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1760.731639] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1760.772589] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1762.243402] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1767.491336] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1776.212621] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1783.684618] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1784.015779] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1787.052203] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1787.117753] Lustre: 59297:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1787.133403] Lustre: 59297:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1787.147949] Lustre: 59297:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1787.158303] Lustre: 59297:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1787.167691] Lustre: 59297:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1787.175644] Lustre: 59297:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1791.786232] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1800.228904] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1805.881173] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1811.127934] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1819.203872] Lustre: server umount lustre-OST0000 complete [ 1833.653803] Lustre: server umount lustre-MDT0000 complete [ 1849.375373] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1853.583347] Key type lgssc unregistered [ 1854.068513] LNet: 60908:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1854.085778] LNetError: 60908:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1855.186819] LNet: Removed LNI 192.168.202.133@tcp [ 1856.531057] Key type .llcrypt unregistered [ 1856.533078] Key type ._llcrypt unregistered [ 1893.451238] Key type ._llcrypt registered [ 1893.454417] Key type .llcrypt registered [ 1893.561667] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 1894.932776] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1894.947955] alg: No test for adler32 (adler32-zlib) [ 1896.334649] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 1896.713555] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 1898.424193] Key type lgssc registered [ 1900.354717] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1910.067411] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1910.104735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1911.643436] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1917.810220] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1926.758686] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1934.045688] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1934.507267] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1936.594570] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1936.757286] Lustre: 63738:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1936.765496] Lustre: 63738:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1936.778554] Lustre: 63738:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1936.786728] Lustre: 63738:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1936.799469] Lustre: 63738:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1936.804189] Lustre: 63738:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1942.522795] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 1950.949416] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1957.571753] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1967.591882] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1967.592785] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1967.625632] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1972.344885] Lustre: server umount lustre-OST0000 complete [ 1977.514296] Lustre: server umount lustre-MDT0000 complete [ 1990.888812] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 1994.043283] Key type lgssc unregistered [ 1994.441695] LNet: 65296:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1994.461929] LNetError: 65296:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1994.491080] LNet: Removed LNI 192.168.202.133@tcp [ 1995.639641] Key type .llcrypt unregistered [ 1995.641714] Key type ._llcrypt unregistered [ 2019.567593] Key type ._llcrypt registered [ 2019.573840] Key type .llcrypt registered [ 2019.807567] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2037.909393] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2039.708882] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2039.725607] alg: No test for adler32 (adler32-zlib) [ 2041.021444] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 2041.541457] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 2043.168243] Key type lgssc registered [ 2044.676050] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2094.451719] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2094.467327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2095.819995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2095.888251] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2096.053974] Lustre: lustre-MDT0000: new disk, initializing [ 2096.185711] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2096.215886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2102.630736] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2114.877255] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2115.049864] Lustre: 68452:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2115.133307] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2115.136720] Lustre: Skipped 1 previous similar message [ 2115.212331] Lustre: lustre-MDT0001: new disk, initializing [ 2115.351986] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2115.423087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2115.438759] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2121.563318] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2131.288581] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2138.988519] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2146.575247] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2146.898353] Lustre: lustre-OST0000: new disk, initializing [ 2146.901662] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2146.909513] Lustre: 70419:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2147.001387] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2154.397634] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2156.169792] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2156.180992] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2156.328793] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2163.135202] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2167.634472] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2167.953250] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2173.584309] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2173.868190] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2176.993286] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2176.994865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2177.000707] LustreError: Skipped 1 previous similar message [ 2177.010599] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2177.017189] Lustre: Skipped 1 previous similar message [ 2177.043650] Lustre: Skipped 1 previous similar message [ 2182.117471] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2182.119473] Lustre: Skipped 1 previous similar message [ 2182.505755] Lustre: server umount lustre-OST0000 complete [ 2187.239384] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2187.240146] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2187.258090] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2192.362540] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2192.373080] Lustre: Skipped 2 previous similar messages [ 2192.942762] Lustre: server umount lustre-MDT0000 complete [ 2197.033097] LustreError: 68443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788439853 with bad export cookie 18078878479997515388 [ 2197.041288] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2197.047404] LustreError: 68443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2197.537773] Lustre: server umount lustre-MDT0001 complete [ 2207.068334] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 08:51:02 (1788439862) [ 2221.885273] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2265.910830] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2277.647191] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2317.757399] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2318.058343] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2318.077622] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2318.130846] Lustre: lustre-MDT0000: new disk, initializing [ 2318.189389] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2318.202643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2322.724419] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2333.789248] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2333.920308] Lustre: 76461:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2333.948987] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2333.950732] Lustre: Skipped 1 previous similar message [ 2333.986270] Lustre: lustre-MDT0001: new disk, initializing [ 2334.036573] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2334.064916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2334.073162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2339.455331] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2351.787105] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2357.624425] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2364.433301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2364.712206] Lustre: lustre-OST0000: new disk, initializing [ 2364.716883] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2364.726801] Lustre: 78430:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2364.828035] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2370.643079] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2370.651352] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2370.805679] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2371.855215] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2381.411359] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2387.062512] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2387.304889] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2393.199363] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2393.463826] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2396.645249] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2396.652242] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2396.659714] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2396.685042] Lustre: Skipped 2 previous similar messages [ 2401.544931] Lustre: server umount lustre-OST0000 complete [ 2405.861945] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2405.882781] LustreError: Skipped 1 previous similar message [ 2405.893954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2405.922938] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2405.934926] Lustre: Skipped 1 previous similar message [ 2411.782628] Lustre: server umount lustre-MDT0000 complete [ 2415.746790] LustreError: 76453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788440071 with bad export cookie 18078878479997516277 [ 2415.778257] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2415.787278] LustreError: 76453:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2416.369099] Lustre: server umount lustre-MDT0001 complete [ 2427.600409] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 08:54:43 (1788440083) [ 2434.826057] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2435.372511] LustreError: 80842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2435.407052] LustreError: 80842:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2435.470787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2440.680791] LustreError: 80843:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2440.799622] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2445.802729] LustreError: 80842:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2449.141571] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2454.921340] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2467.073681] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2476.767411] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2485.788025] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2486.360288] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2486.371991] Lustre: Skipped 1 previous similar message [ 2493.813366] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2501.935943] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2506.601924] Lustre: 83515:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 748c5959-c119-4fe1-898c-fc4794d3e992 at adminstrative request [ 2511.333898] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2511.349269] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2521.573172] Lustre: MGS: Client 748c5959-c119-4fe1-898c-fc4794d3e992 (at 0@lo) reconnecting [ 2521.583571] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xfae5143109b4bd67 to 0xfae5143109b4c007 [ 2521.611494] Lustre: MGC192.168.202.133@tcp: Connection restored to 0@lo (at 0@lo) [ 2545.030247] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2545.287678] Lustre: lustre-OST0001: new disk, initializing [ 2545.291445] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2545.297947] Lustre: 83909:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2552.643594] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2554.543090] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2554.563089] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2554.671282] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2563.190789] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2570.211508] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2570.225613] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2570.240673] Lustre: Skipped 1 previous similar message [ 2570.252605] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2570.262419] Lustre: Skipped 3 previous similar messages [ 2573.377080] Lustre: server umount lustre-OST0001 complete [ 2575.341790] LustreError: 84531:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2575.369032] LustreError: 84531:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2580.465486] LustreError: 84375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2580.468142] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2580.476091] LustreError: 84375:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2580.484247] Lustre: Skipped 2 previous similar messages [ 2583.830351] Lustre: server umount lustre-OST0000 complete [ 2595.815394] LustreError: 80839:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2595.852170] LustreError: 80839:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2596.352893] Lustre: server umount lustre-MDT0000 complete [ 2601.337563] LustreError: 81578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788440257 with bad export cookie 18078878479997517831 [ 2601.348388] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2601.355174] LustreError: 81578:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2601.915746] Lustre: server umount lustre-MDT0001 complete [ 2612.256356] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 08:57:48 (1788440268) [ 2614.442926] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2616.617304] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 08:57:52 (1788440272) [ 2618.941404] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2621.640054] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 08:57:56 (1788440276) [ 2624.267804] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2626.403622] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 08:58:02 (1788440282) [ 2628.413303] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2630.691051] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 08:58:06 (1788440286) [ 2632.746881] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2639.348196] loop: module loaded [ 2639.391698] loop0: detected capacity change from 0 to 4294967296 [ 2639.444383] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2641.281206] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2644.836470] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2649.253636] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2677.092317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2677.151671] LustreError: 88039:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2677.159677] LustreError: 88039:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2677.167513] LustreError: 88039:0:(tgt_mount.c:2585:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2677.177167] LustreError: 88039:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2698.220612] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 2711.148693] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 2750.880667] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2751.092139] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2751.119293] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2751.206777] Lustre: lustre-MDT0000: new disk, initializing [ 2751.326671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2751.335748] Lustre: Skipped 1 previous similar message [ 2751.378230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2756.503696] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2769.207559] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2769.385777] Lustre: 90908:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2769.801453] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2774.922733] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2784.748654] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2790.774904] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2799.115831] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2799.392387] Lustre: 92877:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2807.647564] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2809.456458] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2809.631189] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2817.727779] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2823.198495] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2823.485798] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2829.855787] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2830.182887] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2835.425440] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2835.432692] LustreError: Skipped 2 previous similar messages [ 2835.438604] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2835.451959] Lustre: Skipped 3 previous similar messages [ 2835.457888] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2835.465376] Lustre: Skipped 6 previous similar messages [ 2838.414202] Lustre: server umount lustre-OST0000 complete [ 2848.792416] Lustre: server umount lustre-MDT0000 complete [ 2850.791541] LustreError: 90919:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2850.820887] LustreError: 90919:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2853.422732] LustreError: 90900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788440509 with bad export cookie 18078878479997518447 [ 2853.426508] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2853.440568] LustreError: 90900:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2854.205912] Lustre: server umount lustre-MDT0001 complete [ 2863.076830] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 09:01:58 (1788440518) [ 2870.609293] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2876.339304] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2884.583958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2884.894690] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2884.903105] Lustre: Skipped 3 previous similar messages [ 2889.217775] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2896.248805] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2901.828078] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2910.191059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2917.929386] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2926.215756] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2931.693187] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2931.927289] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2936.886415] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2937.100708] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2945.472025] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2945.809907] Lustre: lustre-OST0001: new disk, initializing [ 2945.827193] Lustre: Skipped 2 previous similar messages [ 2945.831284] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2945.837685] Lustre: Skipped 3 previous similar messages [ 2945.844953] Lustre: 98681:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2951.866087] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2951.874468] Lustre: Skipped 2 previous similar messages [ 2951.879860] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2952.044231] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2953.787849] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 2964.449751] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2970.533091] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2970.741913] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2976.372579] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2976.670783] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2998.873504] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3027.371531] Lustre: Failing over lustre-MDT0000 [ 3027.859358] Lustre: server umount lustre-MDT0000 complete [ 3028.449392] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3028.462398] LustreError: Skipped 1 previous similar message [ 3028.464488] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3028.476320] Lustre: Skipped 2 previous similar messages [ 3028.482637] LustreError: 95304:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3028.513872] LustreError: 95304:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 3045.344148] Lustre: 66672:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788440685/real 1788440685] req@000000000baad155 x1875314944003200/t0(0) o400->MGC192.168.202.133@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1788440701 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3045.369753] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3056.269880] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3056.275552] LDISKFS-fs (dm-0): recovery complete [ 3056.289780] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3070.269021] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3071.680084] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3071.693628] Lustre: 97147:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3074.624245] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3075.554563] Lustre: 95304:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3075.579564] Lustre: 95304:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 3075.608560] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 3075.730174] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3076.057281] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3076.381724] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3076.390045] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3076.720150] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3076.725177] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 3076.745183] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3076.762086] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 3077.425372] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3077.442457] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3077.768380] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3077.773560] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 3078.745095] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3078.758223] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 5 previous similar messages [ 3079.742258] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3079.757835] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3080.080276] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3080.088655] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 3082.987726] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3083.008362] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 3083.986696] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3083.999769] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3084.320332] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3084.336724] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 3091.017258] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3091.037392] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 3092.011356] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3092.036424] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3092.360234] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3092.366036] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 3107.299480] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3107.311521] Lustre: 101536:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 3108.267740] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3108.293410] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3108.616605] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3108.634153] LustreError: 101536:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 3131.208352] Lustre: 101536:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3131.225177] Lustre: 101536:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3131.240839] Lustre: 101536:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 5d8fc7a5-c5aa-4a9f-9ed8-8015c59f36ba@192.168.202.33@tcp [ 3131.274761] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3131.286307] Lustre: 101536:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000ddbae22e x1875314927013120/t0(8589934770) o35->5d8fc7a5-c5aa-4a9f-9ed8-8015c59f36ba@192.168.202.33@tcp:573/0 lens 392/0 e 0 to 0 dl 1788440798 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3131.397388] Lustre: lustre-MDT0000: Denying connection for new client 5d8fc7a5-c5aa-4a9f-9ed8-8015c59f36ba (at 192.168.202.33@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3131.412384] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3131.479269] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3131.580495] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3131.601656] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3131.634056] Lustre: Skipped 2 previous similar messages [ 3131.704180] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 3131.707135] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:46 to 0x2c0000401:545) [ 3149.281985] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3149.286625] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3149.293490] Lustre: Skipped 3 previous similar messages [ 3149.296565] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3149.300050] Lustre: Skipped 2 previous similar messages [ 3155.149187] Lustre: server umount lustre-OST0000 complete [ 3157.481341] LustreError: 101923:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3157.529651] LustreError: 101923:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 56 previous similar messages [ 3165.676988] Lustre: server umount lustre-OST0001 complete [ 3184.096245] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3184.277900] Lustre: server umount lustre-MDT0000 complete [ 3187.973840] LustreError: 95284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788440844 with bad export cookie 18078878479997569715 [ 3187.980183] LustreError: 95284:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3187.982476] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3188.373858] Lustre: server umount lustre-MDT0001 complete [ 3196.022995] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 09:07:32 (1788440852) [ 3205.183790] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3205.191798] Lustre: *** cfs_fail_loc=197, val=0*** [ 3220.768808] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3225.889631] LustreError: 103515:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.133@tcp: failed processing log, type 4: rc = -110 [ 3231.061965] Lustre: *** cfs_fail_loc=197, val=0*** [ 3231.082784] Lustre: Skipped 65 previous similar messages [ 3251.680463] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3251.694394] Lustre: Skipped 3 previous similar messages [ 3259.577675] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3268.003892] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3270.498805] Lustre: server umount lustre-OST0000 complete [ 3280.551759] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 09:08:56 (1788440936) [ 3302.175698] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3302.564320] LustreError: 105136:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3302.590039] LustreError: 105136:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 3307.313702] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3317.567976] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3322.491557] LustreError: 105167:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3322.505795] LustreError: 105167:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3322.521355] LustreError: 105167:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3322.808788] Lustre: server umount lustre-MDT0000 complete [ 3361.239325] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 3373.283907] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3415.109864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3415.406415] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3415.431996] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3415.525605] Lustre: lustre-MDT0000: new disk, initializing [ 3415.624671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3421.091268] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3432.089929] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3432.205288] Lustre: 109360:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3432.472649] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3437.499497] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3447.437596] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3453.170765] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3461.172176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3461.374729] Lustre: 111328:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3469.662872] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3469.758292] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3482.048537] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3489.011719] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3489.510838] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3496.927420] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3497.334951] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3500.513356] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3500.518444] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3500.528927] LustreError: Skipped 3 previous similar messages [ 3500.538706] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3500.541060] Lustre: Skipped 10 previous similar messages [ 3500.622232] Lustre: Skipped 6 previous similar messages [ 3505.657734] Lustre: server umount lustre-OST0000 complete [ 3516.070707] Lustre: server umount lustre-MDT0000 complete [ 3520.020081] LustreError: 109350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441176 with bad export cookie 18078878479997580264 [ 3520.026359] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3520.048695] LustreError: 109350:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3520.601617] Lustre: server umount lustre-MDT0001 complete [ 3530.610550] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 09:13:06 (1788441186) [ 3541.895449] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 3545.200067] Key type lgssc unregistered [ 3545.653141] LNet: 113815:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3545.659981] LNetError: 113815:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3545.696118] LNet: Removed LNI 192.168.202.133@tcp [ 3546.478816] Key type .llcrypt unregistered [ 3546.481747] Key type ._llcrypt unregistered [ 3565.414916] Key type ._llcrypt registered [ 3565.418279] Key type .llcrypt registered [ 3565.560776] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 3582.067936] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3583.636523] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3583.676295] alg: No test for adler32 (adler32-zlib) [ 3584.791535] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 3585.065424] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 3586.768594] Key type lgssc registered [ 3588.263615] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3630.111480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3630.142711] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3631.439333] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3631.469248] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3631.634990] Lustre: lustre-MDT0000: new disk, initializing [ 3631.766787] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3631.805573] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3637.474996] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3648.415477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3648.670209] Lustre: lustre-OST0000: new disk, initializing [ 3648.675496] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3648.679963] Lustre: Skipped 1 previous similar message [ 3648.684251] Lustre: 117984:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3648.795025] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3654.204839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3654.218962] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3654.266128] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3655.950766] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3667.036039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3667.264486] Lustre: lustre-OST0001: new disk, initializing [ 3667.267784] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3667.276883] Lustre: 118849:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3667.369968] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3674.211666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3674.226436] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3674.270899] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3674.899898] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3684.322891] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3684.324484] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3684.345773] Lustre: Skipped 1 previous similar message [ 3684.368282] Lustre: Skipped 1 previous similar message [ 3689.442269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3689.995232] Lustre: server umount lustre-MDT0000 complete [ 3697.090869] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3702.383835] LustreError: 117172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441358 with bad export cookie 6206163336788146859 [ 3702.385547] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3702.394101] LustreError: 117172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3702.730307] Lustre: server umount lustre-OST0000 complete [ 3707.655389] Lustre: server umount lustre-OST0001 complete [ 3716.936607] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 09:16:13 (1788441373) [ 3741.068309] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 09:16:36 (1788441396) [ 3752.259830] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3777.162362] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 09:17:13 (1788441433) [ 3817.559185] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 3828.289539] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 3866.450567] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3866.754938] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3866.814467] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3866.957159] Lustre: lustre-MDT0000: new disk, initializing [ 3867.046774] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3867.067489] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3871.718498] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3883.138158] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3883.279624] Lustre: 126604:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3883.348298] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3883.353943] Lustre: Skipped 1 previous similar message [ 3883.486738] Lustre: lustre-MDT0001: new disk, initializing [ 3883.606501] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3883.674930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3883.678377] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3888.073036] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3897.988558] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3905.608069] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3914.062800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3914.487411] Lustre: lustre-OST0000: new disk, initializing [ 3914.495405] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3914.504336] Lustre: 128574:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3914.579171] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3922.602713] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3924.046297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3924.066656] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3924.241582] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3933.629651] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3938.388744] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3938.644566] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3942.342274] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3942.559599] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3944.930301] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3944.934832] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3944.948430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3944.964753] Lustre: Skipped 1 previous similar message [ 3944.972081] Lustre: Skipped 1 previous similar message [ 3950.052721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3950.063890] Lustre: Skipped 2 previous similar messages [ 3950.767747] Lustre: server umount lustre-OST0000 complete [ 3955.170486] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3955.173377] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3955.184964] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3955.205103] LustreError: Skipped 1 previous similar message [ 3960.989220] Lustre: server umount lustre-MDT0000 complete [ 3965.137637] LustreError: 127400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441621 with bad export cookie 6206163336788148077 [ 3965.153121] LustreError: 127400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3965.154513] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3965.488487] Lustre: server umount lustre-MDT0001 complete [ 3975.464379] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 09:20:31 (1788441631) [ 3981.930938] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3982.730693] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3987.009664] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 3987.942211] LustreError: 130987:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3987.960864] LustreError: 130987:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3992.045122] LustreError: 130988:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3996.542732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4002.549525] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4010.972454] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4017.357791] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4025.217657] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4025.591394] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4025.600755] Lustre: Skipped 1 previous similar message [ 4031.817975] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4040.217027] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4048.122341] Lustre: *** cfs_fail_loc=159, val=0*** [ 4064.517038] Lustre: lustre-MDT0000: Client c73fb770-833c-4ab1-8494-9f465bb369fe (at 192.168.202.33@tcp) reconnecting [ 4066.323957] Lustre: *** cfs_fail_loc=159, val=0*** [ 4066.338658] Lustre: Skipped 6 previous similar messages [ 4081.875691] Lustre: lustre-MDT0000: Client c73fb770-833c-4ab1-8494-9f465bb369fe (at 192.168.202.33@tcp) reconnecting [ 4102.625547] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4102.641381] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4102.653368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4102.661966] Lustre: Skipped 2 previous similar messages [ 4102.667842] Lustre: Skipped 3 previous similar messages [ 4106.141593] Lustre: server umount lustre-OST0000 complete [ 4112.866051] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4112.883773] Lustre: Skipped 1 previous similar message [ 4117.146656] Lustre: server umount lustre-MDT0000 complete [ 4117.988090] LustreError: 130987:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4118.004571] LustreError: 130987:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4121.223433] LustreError: 130969:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788441777 with bad export cookie 6206163336788148966 [ 4121.224651] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4121.234907] LustreError: 130969:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4121.610227] Lustre: server umount lustre-MDT0001 complete [ 4134.295747] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4138.044839] Key type lgssc unregistered [ 4138.440718] LNet: 134765:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4138.450328] LNetError: 134765:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4138.462500] LNet: Removed LNI 192.168.202.133@tcp [ 4139.425956] Key type .llcrypt unregistered [ 4139.433111] Key type ._llcrypt unregistered [ 4167.476366] Key type ._llcrypt registered [ 4167.482288] Key type .llcrypt registered [ 4167.672296] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4169.210295] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4169.372526] alg: No test for adler32 (adler32-zlib) [ 4170.486158] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 4170.663636] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4172.272157] Key type lgssc registered [ 4173.366209] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4181.644371] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4181.664847] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4183.181444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4188.113204] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4193.763148] LustreError: 136175:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4193.780112] LustreError: 136175:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4197.693694] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4198.090937] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4203.589468] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4212.310583] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4219.073663] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4228.799304] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4229.094168] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4237.361630] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4246.037323] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4253.699270] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4266.035662] Lustre: *** cfs_fail_loc=159, val=0*** [ 4281.633118] Lustre: lustre-MDT0000: Client a485a511-b51f-4c8c-9e53-2199fd32afd9 (at 192.168.202.33@tcp) reconnecting [ 4284.122763] Lustre: *** cfs_fail_loc=159, val=0*** [ 4300.525864] Lustre: lustre-MDT0001: Client a485a511-b51f-4c8c-9e53-2199fd32afd9 (at 192.168.202.33@tcp) reconnecting [ 4302.047102] Lustre: *** cfs_fail_loc=159, val=0*** [ 4302.048706] Lustre: Skipped 3 previous similar messages [ 4318.557991] Lustre: lustre-MDT0001: Client a485a511-b51f-4c8c-9e53-2199fd32afd9 (at 192.168.202.33@tcp) reconnecting [ 4334.322747] Lustre: 139266:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4337.968353] Lustre: *** cfs_fail_loc=159, val=0*** [ 4337.972320] Lustre: Skipped 4 previous similar messages [ 4353.691588] Lustre: lustre-MDT0000: Client 74dcee12-d5e3-43a7-9071-4cbc6d7f101c (at 192.168.202.33@tcp) reconnecting [ 4355.543714] Lustre: *** cfs_fail_loc=159, val=0*** [ 4355.564103] Lustre: Skipped 15 previous similar messages [ 4372.187196] Lustre: lustre-MDT0000: Client 74dcee12-d5e3-43a7-9071-4cbc6d7f101c (at 192.168.202.33@tcp) reconnecting [ 4373.933789] Lustre: 139554:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4380.135749] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4380.141394] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4380.162288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4381.667113] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4381.669317] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4381.679067] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4385.496990] Lustre: server umount lustre-OST0000 complete [ 4391.907773] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4391.922470] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4391.925303] Lustre: Skipped 1 previous similar message [ 4395.764701] Lustre: server umount lustre-MDT0000 complete [ 4397.027057] LustreError: 138897:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4397.046279] LustreError: 138897:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4399.829963] LustreError: 136155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442055 with bad export cookie 13016381554342225524 [ 4399.835486] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4399.837318] LustreError: 136155:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4400.131078] Lustre: server umount lustre-MDT0001 complete [ 4413.891527] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4417.620478] Key type lgssc unregistered [ 4417.915745] LNet: 140608:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4417.927731] LNetError: 140608:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4417.954178] LNet: Removed LNI 192.168.202.133@tcp [ 4418.979459] Key type .llcrypt unregistered [ 4418.985097] Key type ._llcrypt unregistered [ 4444.727622] Key type ._llcrypt registered [ 4444.730276] Key type .llcrypt registered [ 4444.863077] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4446.008341] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4446.095936] alg: No test for adler32 (adler32-zlib) [ 4447.200891] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 4447.501317] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4449.213753] Key type lgssc registered [ 4450.411114] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4459.132689] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4459.154636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4460.574782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4464.355202] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4470.759501] LustreError: 142020:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4470.784343] LustreError: 142020:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4471.676903] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4471.948093] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4475.635679] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4483.382733] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4489.571110] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4497.372446] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4497.718343] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4504.823786] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4505.931752] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4512.418506] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4517.274581] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4521.168666] Lustre: 144604:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4524.093727] Lustre: 144652:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4525.467838] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4526.502515] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4530.524389] Lustre: 144799:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4531.507797] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4532.518638] Lustre: 144907:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4537.825022] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4537.834186] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4537.845197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4537.877368] Lustre: Skipped 1 previous similar message [ 4542.949314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4542.956572] Lustre: Skipped 2 previous similar messages [ 4543.307806] Lustre: server umount lustre-OST0000 complete [ 4548.067262] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4548.083563] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4548.092033] Lustre: Skipped 1 previous similar message [ 4553.190123] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4553.198901] Lustre: Skipped 1 previous similar message [ 4554.068789] Lustre: server umount lustre-MDT0000 complete [ 4558.109291] LustreError: 141999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442214 with bad export cookie 4167436721022352907 [ 4558.113799] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4558.116533] LustreError: 141999:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4558.383895] Lustre: server umount lustre-MDT0001 complete [ 4571.599707] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4574.592495] Key type lgssc unregistered [ 4575.017799] LNet: 145962:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4575.025108] LNetError: 145962:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4575.043117] LNet: Removed LNI 192.168.202.133@tcp [ 4575.792312] Key type .llcrypt unregistered [ 4575.794318] Key type ._llcrypt unregistered [ 4602.427892] Key type ._llcrypt registered [ 4602.432811] Key type .llcrypt registered [ 4602.615189] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4604.000388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4604.028177] alg: No test for adler32 (adler32-zlib) [ 4605.465017] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 4605.904809] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4607.632620] Key type lgssc registered [ 4609.476162] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4617.075140] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4617.085617] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4618.571303] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4623.543551] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4627.939918] LustreError: 147375:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4627.960562] LustreError: 147375:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4632.225076] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4632.723625] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4637.905837] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4645.901954] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4651.377203] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4659.330418] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4659.895370] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4662.985198] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4667.230439] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4675.098277] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4677.313190] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4681.787235] Lustre: *** cfs_fail_loc=159, val=0*** [ 4698.371307] Lustre: lustre-MDT0000: Client dd0777ba-30a3-49e6-a38e-863f7b28a7e2 (at 192.168.202.33@tcp) reconnecting [ 4708.324495] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4708.327487] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4708.330412] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4708.341729] Lustre: Skipped 1 previous similar message [ 4710.491865] Lustre: server umount lustre-OST0000 complete [ 4718.562927] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4718.568351] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4718.584596] Lustre: Skipped 1 previous similar message [ 4718.597549] Lustre: Skipped 2 previous similar messages [ 4723.683653] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4723.689484] Lustre: Skipped 1 previous similar message [ 4728.802893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4728.811337] Lustre: Skipped 1 previous similar message [ 4729.313852] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4729.549544] Lustre: server umount lustre-MDT0000 complete [ 4733.506095] LustreError: 147355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442389 with bad export cookie 15667779305914265472 [ 4733.510136] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4733.514934] LustreError: 147355:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4734.020883] Lustre: server umount lustre-MDT0001 complete [ 4747.057330] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4750.253485] Key type lgssc unregistered [ 4750.582716] LNet: 151040:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4750.592509] LNetError: 151040:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4750.607021] LNet: Removed LNI 192.168.202.133@tcp [ 4751.638565] Key type .llcrypt unregistered [ 4751.641603] Key type ._llcrypt unregistered [ 4777.436554] Key type ._llcrypt registered [ 4777.439386] Key type .llcrypt registered [ 4777.614032] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 4778.951800] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4779.000569] alg: No test for adler32 (adler32-zlib) [ 4780.596549] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 4781.003317] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 4782.928290] Key type lgssc registered [ 4784.377271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4793.913239] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4793.966143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4795.679416] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4801.568954] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4805.091250] LustreError: 152450:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4805.125232] LustreError: 152450:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4810.231850] LustreError: 152451:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4811.749935] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4812.258821] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4817.803915] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4825.697681] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4832.080782] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4838.868399] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4839.274759] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4840.341467] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4845.896625] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 4852.759853] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4857.709197] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4877.872949] Lustre: lustre-MDT0001: evicting 362c0331-d735-43af-a1f3-39a593978f66 (at 192.168.202.33@tcp) by administrative request [ 4906.977748] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4906.990509] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4907.014956] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4909.028117] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4909.031956] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4909.048373] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4911.951144] Lustre: server umount lustre-OST0000 complete [ 4919.271250] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4919.306065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4919.336755] Lustre: Skipped 2 previous similar messages [ 4922.439520] Lustre: server umount lustre-MDT0000 complete [ 4924.386896] LustreError: 155148:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4924.400594] LustreError: 155148:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4926.582617] LustreError: 153192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442582 with bad export cookie 2636194859928933534 [ 4926.592268] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4926.603199] LustreError: 153192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4927.098603] Lustre: server umount lustre-MDT0001 complete [ 4938.191983] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 4941.292886] Key type lgssc unregistered [ 4941.624372] LNet: 156519:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4941.638445] LNetError: 156519:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4941.656913] LNet: Removed LNI 192.168.202.133@tcp [ 4942.444858] Key type .llcrypt unregistered [ 4942.447878] Key type ._llcrypt unregistered [ 5015.485481] Key type ._llcrypt registered [ 5015.487295] Key type .llcrypt registered [ 5015.577512] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5017.107302] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5017.166258] alg: No test for adler32 (adler32-zlib) [ 5018.365369] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 5018.596817] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5020.248273] Key type lgssc registered [ 5021.632953] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5029.094638] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5029.119278] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5030.774055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5036.225719] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5040.163355] LustreError: 159007:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5040.199522] LustreError: 159007:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5045.222060] LustreError: 159006:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5046.861408] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5047.597056] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5052.871040] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5060.277816] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5065.853131] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5072.073633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5072.294050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5073.335612] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5075.391636] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5078.888910] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5087.060145] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5098.981103] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5098.988257] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5099.021300] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5101.032134] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5101.033884] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5101.050511] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5101.072718] Lustre: Skipped 1 previous similar message [ 5104.117991] Lustre: server umount lustre-OST0000 complete [ 5108.711641] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5108.729393] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5108.744148] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5111.267536] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5111.273517] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5111.287179] Lustre: Skipped 1 previous similar message [ 5114.313841] Lustre: server umount lustre-MDT0000 complete [ 5116.385716] LustreError: 159025:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5116.410682] LustreError: 159025:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 5117.774898] LustreError: 158987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788442773 with bad export cookie 10192208095108496871 [ 5117.781580] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5117.793292] LustreError: 158987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5118.238456] Lustre: server umount lustre-MDT0001 complete [ 5130.266707] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 5133.545342] Key type lgssc unregistered [ 5133.876593] LNet: 162548:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5133.883835] LNetError: 162548:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5133.909302] LNet: Removed LNI 192.168.202.133@tcp [ 5134.877753] Key type .llcrypt unregistered [ 5134.881273] Key type ._llcrypt unregistered [ 5160.361174] Key type ._llcrypt registered [ 5160.363973] Key type .llcrypt registered [ 5160.510306] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5161.760108] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5161.919643] alg: No test for adler32 (adler32-zlib) [ 5163.009059] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 5163.260604] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5164.920245] Key type lgssc registered [ 5166.334601] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5182.331493] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 09:40:38 (1788442838) [ 5198.943847] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 5209.456586] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5246.374304] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5246.414734] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5247.664750] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5247.702440] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5247.792887] Lustre: lustre-MDT0000: new disk, initializing [ 5247.894943] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5247.926280] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5252.976848] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5263.959868] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5264.096253] Lustre: 166808:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5264.150914] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5264.155942] Lustre: Skipped 1 previous similar message [ 5264.255965] Lustre: lustre-MDT0001: new disk, initializing [ 5264.411675] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5264.502161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5264.525746] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5269.512357] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5279.822912] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5285.790723] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5292.743675] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5293.040935] Lustre: lustre-OST0000: new disk, initializing [ 5293.045228] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5293.052732] Lustre: 168778:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5293.162478] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5299.751719] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5300.330731] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5300.346201] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5300.504331] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5309.295370] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5321.190995] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5321.192239] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5321.198616] LustreError: Skipped 1 previous similar message [ 5321.200095] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5321.235098] Lustre: Skipped 1 previous similar message [ 5325.947456] Lustre: server umount lustre-OST0000 complete [ 5330.919800] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5330.922865] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5330.951671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5330.954372] Lustre: Skipped 1 previous similar message [ 5336.548079] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5336.559058] Lustre: Skipped 3 previous similar messages [ 5341.669110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5341.678128] Lustre: Skipped 1 previous similar message [ 5344.224221] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5344.376375] Lustre: server umount lustre-MDT0000 complete [ 5347.885428] LustreError: 166803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788443004 with bad export cookie 11701780536340123698 [ 5347.891028] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5347.899618] LustreError: 166803:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5348.306949] Lustre: server umount lustre-MDT0001 complete [ 5359.868443] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 5364.411756] Key type lgssc unregistered [ 5364.967951] LNet: 170685:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5364.972200] LNetError: 170685:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5365.999660] LNet: Removed LNI 192.168.202.133@tcp [ 5366.952563] Key type .llcrypt unregistered [ 5366.958435] Key type ._llcrypt unregistered [ 5395.759254] Key type ._llcrypt registered [ 5395.761391] Key type .llcrypt registered [ 5395.878230] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 5397.258670] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5397.276169] alg: No test for adler32 (adler32-zlib) [ 5398.354595] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 5398.653744] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 5400.448170] Key type lgssc registered [ 5401.959054] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5410.244849] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5410.260870] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5411.578782] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5415.078868] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5421.035539] LustreError: 172110:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5421.048959] LustreError: 172110:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 5423.929510] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5424.273922] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5429.562929] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5438.691863] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5445.078064] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5452.018712] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5452.437201] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5457.592859] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5459.627445] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 5467.859626] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5474.741912] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5474.763544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5474.785869] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5474.796974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.822563] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5475.837638] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5475.867134] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5475.886962] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5475.898604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.916331] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5476.949587] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5476.959985] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5476.965386] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5476.968549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.978503] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5477.522581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.264595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.159154] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5479.165338] Lustre: Skipped 2 previous similar messages [ 5479.187447] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5479.196479] Lustre: Skipped 2 previous similar messages [ 5479.198693] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5479.201131] Lustre: Skipped 2 previous similar messages [ 5479.203540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.209457] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5479.214771] Lustre: Skipped 2 previous similar messages [ 5479.843915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.415750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.121706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.796303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.971875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.007608] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5484.013687] Lustre: Skipped 5 previous similar messages [ 5484.030321] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5484.038316] Lustre: Skipped 5 previous similar messages [ 5484.042961] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5484.049813] Lustre: Skipped 5 previous similar messages [ 5484.053282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.060711] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5484.064120] Lustre: Skipped 5 previous similar messages [ 5485.260394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.445817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.661162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.509662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.549644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.281163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.156471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.042421] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5492.044406] Lustre: Skipped 7 previous similar messages [ 5492.049842] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5492.053870] Lustre: Skipped 7 previous similar messages [ 5492.057089] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5492.061303] Lustre: Skipped 7 previous similar messages [ 5492.063766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.068860] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5492.076822] Lustre: Skipped 7 previous similar messages [ 5492.976532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.082402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.145513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.100402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.111361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.055818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.314544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.404345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.574226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.677910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.722828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.660378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.600161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.709902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.589905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.292434] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5508.300397] Lustre: Skipped 15 previous similar messages [ 5508.312562] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5508.320399] Lustre: Skipped 15 previous similar messages [ 5508.322699] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5508.324984] Lustre: Skipped 15 previous similar messages [ 5508.329831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.336349] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5508.338717] Lustre: Skipped 15 previous similar messages [ 5509.181805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.060253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.094198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.037666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.930823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.954727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.663888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.617167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.827410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.746688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.536506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.355933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.302108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.104872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.821087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.885937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.895941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.231371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.777966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.118534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.334517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.813513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.036320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.192366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.207678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.409974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.408306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.836564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.257863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.759371] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5540.765593] Lustre: Skipped 29 previous similar messages [ 5540.794698] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5540.822973] Lustre: Skipped 29 previous similar messages [ 5540.831112] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5540.837545] Lustre: Skipped 29 previous similar messages [ 5540.844962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.855419] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5540.869423] Lustre: Skipped 29 previous similar messages [ 5542.466550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.821632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.118537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.312732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.001974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.727144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.517764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.558617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.850717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.215921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.343899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.655394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.938891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.672739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.569719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.470227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.729731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.926801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.753793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.671981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.571264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.562547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.564412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.896918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.231577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.343952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.334859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.409529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.489384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.637146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.845913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.897389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.033449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.093544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.956124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.030094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.023725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.768579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.493052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.343866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.200788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.199421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.989708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.106569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.011920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.819711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.651094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.400116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.181455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.052197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.867615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.321799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.198907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.505182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.758749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.893512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.904876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.064223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.949904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.871451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.887826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.216910] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5605.226970] Lustre: Skipped 61 previous similar messages [ 5605.262554] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5605.276039] Lustre: Skipped 61 previous similar messages [ 5605.284042] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5605.286629] Lustre: Skipped 61 previous similar messages [ 5605.302030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.314098] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5605.316582] Lustre: Skipped 61 previous similar messages [ 5606.484702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.513562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.910323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.064971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.206683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.466838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.584733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.275141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.090564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.777904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.497495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.267721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.948434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.557384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.211470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.949949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.813564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.755794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.406745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.127429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.720275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.255575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.958673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.893588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.960466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.882392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.847635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.839914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.537821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.472481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.348531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.456377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.318482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.081858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.783447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.515351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.271486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.233096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.289445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.051642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.848632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.571448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.570899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.577832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.533762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.358502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.531509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.547324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.831285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.815154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.813317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.402429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.187962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.477694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.693504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.797258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.861951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.509073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.427900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.417810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.577655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.711350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.936341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.222877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.381661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.449099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.520281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.606899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.748169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.018795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.005739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.191362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.348831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.571923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.749791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.692862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.718191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.645439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.570236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.603639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.591079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.335575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.135970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.761334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.600600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.516383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.483029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.222480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.261947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.007339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.821502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.742735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.692544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.509482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.412051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.250780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.154687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.058985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.883638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.567201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.370227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.156416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.060944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.075499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.907178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.645065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.520241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.528784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.332590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.221560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.112790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.045841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.927703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.006524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.813303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.856262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.873722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.736202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.659128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.452874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.359778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.287186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.214772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.252797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.326463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.334708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.330574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.373662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.394656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.383636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.300234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.271431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.466571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.327128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.196895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.138776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.371184] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5733.383367] Lustre: Skipped 136 previous similar messages [ 5733.430489] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5733.452580] Lustre: Skipped 136 previous similar messages [ 5733.460155] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5733.464746] Lustre: Skipped 136 previous similar messages [ 5733.469453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.482894] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5733.491323] Lustre: Skipped 136 previous similar messages [ 5734.648503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.469519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.429951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.411767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.375607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.478836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.678794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.899637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.195287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.209573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.344092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.338653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.440922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.375685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.321220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.237649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.458443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.255842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.155779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.967293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.946037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.949932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.961990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.957036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.002695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.091964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.116561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.443966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.404464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.380292] LustreError: 172130:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5764.390101] LustreError: 172130:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5764.405625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.345672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.367136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.320871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.499393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.546963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.635868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.602133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.717737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.666534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.922566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.928539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.969529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.087493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.357270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.380177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.424583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.583247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.744599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.990217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.802582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.670817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.950322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.108265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.050263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.132244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.096519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.915702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.840183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.995678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.888082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.904721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.737283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.700942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.650962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.631077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.417813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.380815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.273813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.132896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.104750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.104314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.074803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.887034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.921236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.868479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.063404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.045493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.260412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.062337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.898649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.672975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.568055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.471601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.458593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.271317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.377108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.339310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.408726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.397835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.452737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.343518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.226739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.490272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.294133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.269308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.300516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.116355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.994660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.022724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.814470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.607707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.352574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.287294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.170334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.265191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.112600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.784193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.639622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.434237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.360548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.226684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.269673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.145864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.227327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.663476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.495188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.263577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.168360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.153870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.354957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.330572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.531658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.697393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.854379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.688721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.138853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.350917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.164434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.175927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.975720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.660523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.413243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.086962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.780221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.567901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.312441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.894254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.411156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.883516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.485180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.202158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.054506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.793498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.688064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.713176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.609277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.325577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.205834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.081761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.818513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.528356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.435415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.249732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.140804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.233346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.204758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.194828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.942920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.640534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.421326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.116966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.036224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.974749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.872781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.611713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.370503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.108540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.985949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.884560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.684597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.591624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.462074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.252437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.100048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.183352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.113056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.117160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.019554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.057633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.801692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.615571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.661888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.627711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.707461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.643664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.936848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.974970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.210945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.360113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.677658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.894915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.087171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.196851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.270118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.632281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.898506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.182880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.919639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.189723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.723308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.165937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.297752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.180757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.123088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.865169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.823852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.445303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.344457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.196808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.089150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.918305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.702468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.865231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.731684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.725975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.736056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.689761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.655985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.670478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.652753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.559586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.703358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.921094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.707596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.778531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.682884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.442139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.255741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.102329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.694466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.555801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.463130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.341947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.008577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.863586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.727181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.448945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.028475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.865242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.782440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.765369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.777711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.737833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.746429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.892534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.090734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.064573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.219923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.295167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.291391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.422220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.576417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.710757] LustreError: 172130:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5976.722698] LustreError: 172130:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5976.725494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.771705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.659224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.566485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.369177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.407929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.331509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.365188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.695625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.731906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.875544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.886987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.955989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.890878] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5989.895358] Lustre: Skipped 265 previous similar messages [ 5989.911380] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5989.918690] Lustre: Skipped 265 previous similar messages [ 5989.922069] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5989.926752] Lustre: Skipped 265 previous similar messages [ 5989.930776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.942642] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5989.945415] Lustre: Skipped 265 previous similar messages [ 5991.028129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.838957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.576355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.271133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.167800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.948204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.744388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.595246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.480132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.389628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.290864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.978183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.886202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.743648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.825850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.733617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.845726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.662364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.618874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.627404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.679955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.749820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.833127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.846449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.845898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.764567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.576341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.562981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.307946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.205609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.163231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.131791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.010790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.820196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.677993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.509902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.256299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.007275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.872966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.866568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.842676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.944254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.815084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.796930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.627655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.679198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.491284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.386914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.220705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.201053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.956953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.830650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.705674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.493483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.318225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.134949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.989344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.724601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.496465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.357962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.145487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.117560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.813919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.526291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.236557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.107606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.882241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.647471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.496928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.531189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.698696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.446622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.602177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.660394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.548649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.261362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.936713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.671497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.554136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.744337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.475245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.589527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.737927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.519987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.517709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.155375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.696767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.216816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.806390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.232854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.326697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.667775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.458796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.869197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.015801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.964726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.706466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.278060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.329686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.482624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.774667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.920821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.094771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.876134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.171360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.233088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.078379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.095048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.250552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.958755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.160621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.388716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.516771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.755177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.903095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.836344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.775381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.773347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.851480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.765653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.927222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.609487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.248582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.763252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.193754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.182608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.212558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.031159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.999417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.884307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.860767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.789868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.795504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.608606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.693361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.668327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.666020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.503795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.546664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.484363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.469957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.602423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.716431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.810805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.856797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.777244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.756893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.646553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.952664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.955364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.033861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.935972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.016651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.881224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.801428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.732854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.652891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.545928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.582564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.433817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.258098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.395801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.302543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.481673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.287068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.204593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.157649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.165699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.946381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.011688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.928733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.850820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.624474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.459159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.452742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.427943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.546529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.864070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.879202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.032762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.186342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.605910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.776112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.953602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.518972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.468880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.546908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.582802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.511973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.446545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.344602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.360470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.298446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.148340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.322475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.353101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.232635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.294937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.256532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.448353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.347834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.177499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.519082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.830732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.198113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.586606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.286789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.127022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.152329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.334855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.717145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.621595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.789523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.983036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.916237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.851917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.020059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.036241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.003176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.998575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.913889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.748211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.510242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.227274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.298929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.286726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.272504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.306127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.282428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.240243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.987908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.778977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.515156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.599705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.458493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.455025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.266563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.937237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.697619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.468202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.339853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.327732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.263056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.940950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.651898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.377633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.292188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.299453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.330664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.404409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.392167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.260449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.131867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.951837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.970557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.866241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.799475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.516622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.445531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.320966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.442812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.553622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.704996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.603694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.500443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.342762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.368560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.358357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.206392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.471718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.464806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.308543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.198454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.273774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.356798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.418704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.078655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.305537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.468662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.359560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.443497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.183374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.606796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.416962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.648473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.220776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.526419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.861445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.193258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.317480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.255912] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.045293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.416291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.439922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.415719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.268740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.330128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.131449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.329687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.179367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.263446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.165393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.978792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.307081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.542271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.719693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.641167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.509974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.387280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.441462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.661644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.579279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.650335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.770756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.823815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.044625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.984918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.879233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.866848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.983277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.093768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.174237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.119106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.986892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.009309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.842639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.771482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.688469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.711599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.743152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.885507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.105610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.316051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.594388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.778707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.110871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.103587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.076052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.118797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.279877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.292920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.385306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.560277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.588745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.531291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.424465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.374722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.321335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.320914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.316628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.360086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.183241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.213252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.087724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.102261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.145810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.962662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.896589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.928373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.886840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.867490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.919935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.720910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.602195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.417747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.313247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.194175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.085144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.176273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.964744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.969364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.954122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.828672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.963308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.946156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.015107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.991105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.842944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.806958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.643814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.562762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.425213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.447635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.337661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.300971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.244546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.297692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.196189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.265206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.088362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.144355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.130511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.048834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.890812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.853834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.704447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.840514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.888648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.764250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.741576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.853793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.899619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.688399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.733771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.653769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.557310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.258287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.192156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.231166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.014492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.082398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.903505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.112361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.416393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.736656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.972694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.932823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.275693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.276490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.475896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.913079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.104770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.437670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.382419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.749856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.801274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.594490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.418401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.526365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.488163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.722057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.712553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.033927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.117118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.075472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.118625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.440101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.550028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.976633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.005863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.404190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.517839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.680193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.920211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.450183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.995412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.785585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.793406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.902234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.138321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.386801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.689601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.791655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.002906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.099975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.158742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.014099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.872255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.042174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.215665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.405785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.704731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.600034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.682224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.594458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6461.589700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.666075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.779369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.683352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6799.793780] LustreError: 173932:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6799.803771] LustreError: 172130:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7204.833461] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7204.840593] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7204.860225] Lustre: Skipped 469 previous similar messages [ 7204.868488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7205.863866] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7205.865570] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7210.347494] Lustre: server umount lustre-OST0000 complete [ 7216.107328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7216.115598] Lustre: Skipped 2 previous similar messages [ 7216.117068] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7220.739956] Lustre: server umount lustre-MDT0000 complete [ 7221.217905] LustreError: 172105:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7221.242724] LustreError: 172105:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7224.415059] LustreError: 172091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788444880 with bad export cookie 14167906409063480640 [ 7224.421542] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7224.430345] LustreError: 172091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7224.902478] Lustre: server umount lustre-MDT0001 complete [ 7236.570954] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 7239.454399] Key type lgssc unregistered [ 7239.790568] LNet: 227343:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7239.809851] LNetError: 227343:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7239.847061] LNet: Removed LNI 192.168.202.133@tcp [ 7240.660479] Key type .llcrypt unregistered [ 7240.662911] Key type ._llcrypt unregistered [ 7265.702464] Key type ._llcrypt registered [ 7265.704985] Key type .llcrypt registered [ 7265.821427] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 7266.900681] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7266.925069] alg: No test for adler32 (adler32-zlib) [ 7268.214576] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 7268.444892] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 7270.208130] Key type lgssc registered [ 7271.922178] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7281.917114] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7281.965665] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7283.927211] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7291.914144] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7293.414487] LustreError: 228755:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7293.437318] LustreError: 228755:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7298.532120] LustreError: 228756:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7301.758905] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7302.509368] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7309.171412] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7316.730670] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7322.971228] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7329.927567] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7330.440491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7340.206124] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7351.521589] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7352.950218] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7361.340141] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7363.597954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7363.614244] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7363.618321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7363.629396] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7363.635113] LustreError: 228775:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7367.910779] Lustre: lustre-OST0000: Not available for connect from 192.168.202.33@tcp (stopping) [ 7368.684377] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7368.687209] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7368.693234] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7368.700026] Lustre: Skipped 1 previous similar message [ 7372.337673] Lustre: server umount lustre-OST0000 complete [ 7383.728129] LustreError: 228775:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7407.063642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7407.442691] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7408.488594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.503253] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.510304] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7414.738719] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7422.897173] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7424.601463] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7429.842121] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9214491f3000.ost_server_uuid 50 [ 7431.149426] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9214491f3000.ost_server_uuid in IDLE state after 0 sec [ 7436.769497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7436.778010] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7436.801712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7436.805836] Lustre: Skipped 1 previous similar message [ 7439.329656] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7439.331178] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7439.336617] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7441.014549] Lustre: server umount lustre-OST0000 complete [ 7445.987065] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7445.996469] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7446.019228] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7446.033740] Lustre: Skipped 1 previous similar message [ 7451.569519] Lustre: server umount lustre-MDT0000 complete [ 7454.690047] LustreError: 229485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7454.721965] LustreError: 229485:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7455.814795] LustreError: 233018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445111 with bad export cookie 15534331452163982037 [ 7455.820275] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7455.827660] LustreError: 233018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7456.328314] Lustre: server umount lustre-MDT0001 complete [ 7467.828534] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 7471.285397] Key type lgssc unregistered [ 7471.532036] LNet: 233970:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7471.543165] LNetError: 233970:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7471.571167] LNet: Removed LNI 192.168.202.133@tcp [ 7472.710250] Key type .llcrypt unregistered [ 7472.713487] Key type ._llcrypt unregistered [ 7512.060494] Key type ._llcrypt registered [ 7512.062555] Key type .llcrypt registered [ 7512.206236] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 7514.144332] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7514.219290] alg: No test for adler32 (adler32-zlib) [ 7515.437485] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 7515.793774] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 7517.600180] Key type lgssc registered [ 7519.044696] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7528.707277] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7528.746129] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7530.403630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7535.481988] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7540.717261] LustreError: 235700:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7540.740420] LustreError: 235700:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7545.165046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7545.507604] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7550.561902] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7553.851134] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7560.725050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7561.278555] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7567.886601] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7575.677344] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7575.949751] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7576.132140] Lustre: lustre-OST0001: new disk, initializing [ 7576.148520] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7576.160262] Lustre: 238008:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7576.236817] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7581.666759] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7582.371023] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7582.398070] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7582.659157] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7595.632650] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7603.462745] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7616.339917] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [ 7616.362231] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7616.380933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7618.534077] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7618.549620] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7618.582171] Lustre: Skipped 1 previous similar message [ 7618.586128] Lustre: Skipped 1 previous similar message [ 7622.969284] Lustre: server umount lustre-MDT0000 complete [ 7623.665335] LustreError: 237206:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7623.696168] LustreError: 237206:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7626.448877] LustreError: 238952:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445282 with bad export cookie 8499643043011926802 [ 7626.459993] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7626.464044] LustreError: 236438:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200001b72:0x1:0x0]: rc = -5 [ 7626.900706] Lustre: server umount lustre-MDT0001 complete [ 7642.107749] Lustre: server umount lustre-OST0000 complete [ 7643.944882] Lustre: 234961:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788445284/real 1788445284] req@ffff8e0c385b4e00 x1875320683723264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788445300 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7643.980888] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7643.997933] Lustre: Skipped 1 previous similar message [ 7646.919054] Lustre: server umount lustre-OST0001 complete [ 7668.488504] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 7678.062669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7678.428424] LustreError: 241115:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7678.498082] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7682.255613] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7683.554160] LustreError: 241116:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7688.679220] LustreError: 241115:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7690.237767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7690.654208] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7694.915891] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7698.625232] Lustre: 242257:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7707.696075] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7708.356696] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7715.519258] LustreError: 242610:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7715.559280] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7716.471939] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7723.007432] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7726.417536] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7733.140506] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7740.880937] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7744.573286] Lustre: 244128:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7757.283848] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7757.299049] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7757.320774] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7757.331759] Lustre: Skipped 2 previous similar messages [ 7760.926953] Lustre: server umount mylustre-MDT0000 complete [ 7762.949243] LustreError: 241115:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7762.978884] LustreError: 241115:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7764.478839] LustreError: 241093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445420 with bad export cookie 8499643043011929665 [ 7764.486828] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7764.488807] LustreError: 241093:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 7764.972157] Lustre: server umount mylustre-MDT0001 complete [ 7780.096485] Lustre: server umount mylustre-OST0000 complete [ 7783.905123] Lustre: 234962:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788445424/real 1788445424] req@ffff8e0b039ae680 x1875320683785600/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788445440 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7783.926769] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7783.939396] Lustre: Skipped 3 previous similar messages [ 7785.847237] Lustre: server umount mylustre-OST0001 complete [ 7817.052745] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 7830.196174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7830.842163] LustreError: 246436:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7831.018444] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7831.029106] Lustre: Skipped 1 previous similar message [ 7836.715896] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7846.696952] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7851.735293] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7855.682369] Lustre: 247578:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7862.906878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7871.893620] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7872.500810] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [ 7876.632777] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7882.327730] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7887.896309] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 7889.868250] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7897.706225] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7901.464395] Lustre: 249451:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7913.449088] Lustre: tfs-MDT0000-lwp-OST0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7913.457047] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7913.478269] Lustre: Skipped 3 previous similar messages [ 7913.499757] Lustre: Skipped 6 previous similar messages [ 7916.444663] Lustre: server umount tfs-MDT0000 complete [ 7918.575187] LustreError: 246433:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7918.595808] LustreError: 246433:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 11 previous similar messages [ 7920.616715] LustreError: 246414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445576 with bad export cookie 8499643043011931618 [ 7920.624223] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7920.630609] LustreError: 246414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7920.953943] Lustre: server umount tfs-MDT0001 complete [ 7935.947508] Lustre: server umount tfs-OST0000 complete [ 7939.049104] Lustre: 234963:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788445579/real 1788445579] req@ffff8e0c04025880 x1875320683841024/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788445595 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7939.094643] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7940.723322] Lustre: server umount tfs-OST0001 complete [ 7967.959351] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 7980.511740] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7981.244069] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7981.254029] Lustre: Skipped 3 previous similar messages [ 7985.798092] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 7995.443326] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8001.217278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8004.764727] Lustre: 252846:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8012.218740] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8020.053037] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [ 8020.490864] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8026.103903] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8029.968573] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8035.320651] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8038.012995] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8046.124894] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8050.309565] Lustre: 254717:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8057.315407] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 10:28:33 (1788445713) [ 8076.261115] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8076.271443] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8076.306378] Lustre: Skipped 2 previous similar messages [ 8076.324992] Lustre: Skipped 3 previous similar messages [ 8081.148771] Lustre: server umount lustre-MDT0000 complete [ 8081.385612] LustreError: 251698:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8081.400050] LustreError: 251698:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 10 previous similar messages [ 8085.679166] LustreError: 252447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445741 with bad export cookie 8499643043011933536 [ 8085.690205] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8085.697882] LustreError: 252447:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8086.498851] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8086.519400] Lustre: Skipped 1 previous similar message [ 8092.212684] Lustre: server umount lustre-MDT0001 complete [ 8102.295922] Lustre: server umount lustre-OST0000 complete [ 8120.289280] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8120.515924] Lustre: server umount lustre-OST0001 complete [ 8128.907730] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 8140.460278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 8183.881782] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8184.065284] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8184.086921] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8184.162512] Lustre: lustre-MDT0000: new disk, initializing [ 8184.250959] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 8184.253344] Lustre: Skipped 3 previous similar messages [ 8184.266783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8188.888473] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8199.960473] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8200.096116] Lustre: 257901:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8200.182239] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8200.198744] Lustre: Skipped 1 previous similar message [ 8200.304546] Lustre: lustre-MDT0001: new disk, initializing [ 8200.454981] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8200.475254] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8204.961764] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8215.789504] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8216.035369] Lustre: lustre-OST0000: new disk, initializing [ 8216.038674] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8216.044880] Lustre: 259485:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8217.762315] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8217.774575] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8217.927218] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8222.916293] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8233.838684] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8234.021417] Lustre: lustre-OST0001: new disk, initializing [ 8234.024937] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8234.033729] Lustre: 260361:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8236.020431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8236.041246] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8236.127334] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8240.437581] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8252.535746] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 10:31:48 (1788445908) [ 8266.637504] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 10:32:02 (1788445922) [ 8284.642709] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8284.648179] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8284.669418] LustreError: Skipped 1 previous similar message [ 8284.674366] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8284.683305] Lustre: Skipped 4 previous similar messages [ 8284.700219] Lustre: Skipped 3 previous similar messages [ 8288.634641] Lustre: server umount lustre-OST0000 complete [ 8298.688992] Lustre: server umount lustre-MDT0000 complete [ 8302.387277] LustreError: 257895:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788445958 with bad export cookie 8499643043011935314 [ 8302.388736] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8302.395965] LustreError: 257895:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8308.923343] Lustre: server umount lustre-MDT0001 complete [ 8319.266306] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 8394.917952] Lustre: server umount lustre-OST0001 complete [ 8402.161372] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 8418.989201] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 8461.421631] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8461.721993] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8461.758481] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8461.804713] Lustre: lustre-MDT0000: new disk, initializing [ 8461.899699] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8461.907798] Lustre: Skipped 3 previous similar messages [ 8461.922678] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8466.762387] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8478.289498] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8478.470048] Lustre: 265212:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8478.523784] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8478.526650] Lustre: Skipped 1 previous similar message [ 8478.575558] Lustre: lustre-MDT0001: new disk, initializing [ 8478.675533] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8478.687572] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8483.831453] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8494.405813] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8500.548163] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8507.894956] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8508.292431] Lustre: 267180:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8510.286134] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8510.485805] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8515.580151] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8525.007660] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8536.037259] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8536.048391] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8536.067982] Lustre: Skipped 4 previous similar messages [ 8536.078939] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8536.090230] Lustre: Skipped 11 previous similar messages [ 8537.487470] Lustre: server umount lustre-OST0000 complete [ 8545.260396] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8545.273773] LustreError: Skipped 1 previous similar message [ 8547.754986] Lustre: server umount lustre-MDT0000 complete [ 8551.396314] LustreError: 265219:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8551.421981] LustreError: 265219:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 9 previous similar messages [ 8551.924519] LustreError: 266007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446208 with bad export cookie 8499643043011936784 [ 8551.932405] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8551.935256] LustreError: 266007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8552.398885] Lustre: server umount lustre-MDT0001 complete [ 8563.801798] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 8566.914797] Key type lgssc unregistered [ 8567.187337] LNet: 268940:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8567.194242] LNetError: 268940:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8567.204670] LNet: Removed LNI 192.168.202.133@tcp [ 8568.084526] Key type .llcrypt unregistered [ 8568.088296] Key type ._llcrypt unregistered [ 8595.789338] Key type ._llcrypt registered [ 8595.791431] Key type .llcrypt registered [ 8595.912274] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 8597.142429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8597.227855] alg: No test for adler32 (adler32-zlib) [ 8598.361702] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 8598.610232] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 8600.272162] Key type lgssc registered [ 8601.489813] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8609.186914] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8609.208066] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8610.765109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8615.254121] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8620.016695] LustreError: 270399:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8620.042112] LustreError: 270399:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8623.992043] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8624.395647] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8629.832729] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8637.744511] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8643.334754] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8650.181598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8650.670917] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8656.846155] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8657.729577] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8665.981379] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8667.342183] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8672.225962] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8672.245721] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8672.271639] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8674.279141] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8674.285402] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8674.289949] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8674.317754] Lustre: Skipped 1 previous similar message [ 8675.622691] Lustre: server umount lustre-OST0000 complete [ 8680.930167] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8680.935227] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8680.954232] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8684.517553] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8684.519151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8684.547751] Lustre: Skipped 1 previous similar message [ 8689.633220] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8689.642153] Lustre: Skipped 1 previous similar message [ 8694.242939] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8694.573291] Lustre: server umount lustre-MDT0000 complete [ 8694.762733] LustreError: 270399:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8694.777458] LustreError: 270399:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8699.061281] LustreError: 271135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446355 with bad export cookie 13136563399004435702 [ 8699.075375] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8699.085469] LustreError: 271135:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8699.902084] Lustre: server umount lustre-MDT0001 complete [ 8712.607228] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 8716.289229] Key type lgssc unregistered [ 8716.656981] LNet: 273989:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8716.661647] LNetError: 273989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8716.685542] LNet: Removed LNI 192.168.202.133@tcp [ 8717.952883] Key type .llcrypt unregistered [ 8717.957260] Key type ._llcrypt unregistered [ 8737.758839] Key type ._llcrypt registered [ 8737.761560] Key type .llcrypt registered [ 8737.922273] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 8738.898381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8738.937336] alg: No test for adler32 (adler32-zlib) [ 8740.106213] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 8740.462412] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 8742.272273] Key type lgssc registered [ 8743.695547] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8753.726865] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8753.738416] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8769.248446] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8774.368275] LustreError: 275179:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.202.133@tcp: failed processing log, type 4: rc = -110 [ 8801.057912] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8807.578363] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 8816.373420] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8819.169968] Lustre: server umount lustre-OST0000 complete [ 8835.466455] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 8838.381435] Key type lgssc unregistered [ 8838.690657] LNet: 276810:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8838.700719] LNetError: 276810:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8838.723539] LNet: Removed LNI 192.168.202.133@tcp [ 8839.739480] Key type .llcrypt unregistered [ 8839.744503] Key type ._llcrypt unregistered [ 8881.715162] Key type ._llcrypt registered [ 8881.716947] Key type .llcrypt registered [ 8881.852854] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 8883.194599] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8883.230869] alg: No test for adler32 (adler32-zlib) [ 8884.520676] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 8884.745049] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 8886.448212] Key type lgssc registered [ 8888.391138] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8903.138912] loop0: detected capacity change from 0 to 204800000 [ 8903.334431] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8903.350465] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8903.647659] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8904.799391] loop0: detected capacity change from 0 to 204800000 [ 8905.046336] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8905.057390] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8905.576182] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8906.722807] loop0: detected capacity change from 0 to 204800000 [ 8907.008728] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8907.023418] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8907.177912] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8908.611300] loop0: detected capacity change from 0 to 204800000 [ 8908.900801] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8908.935176] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8909.043664] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8910.174344] loop0: detected capacity change from 0 to 204800000 [ 8910.221391] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8914.792928] loop0: detected capacity change from 0 to 204800000 [ 8914.842577] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8919.325833] loop0: detected capacity change from 0 to 204800000 [ 8919.385901] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8924.099991] loop0: detected capacity change from 0 to 204800000 [ 8924.127856] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8929.429789] loop0: detected capacity change from 0 to 204800000 [ 8929.541369] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8929.616920] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8929.770721] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8929.787836] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8929.806795] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8929.824650] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8929.840118] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8929.861642] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8929.878289] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8929.896333] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8929.916414] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8929.927383] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8929.948474] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8929.966910] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8929.984930] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8930.000590] Lustre: 280223:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8936.629126] Lustre: server umount MGS complete [ 8937.382950] loop0: detected capacity change from 0 to 204800000 [ 8937.472532] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8938.014759] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8938.080924] Lustre: lustre-MDT0000: Aborting client recovery [ 8939.157887] loop1: detected capacity change from 0 to 204800000 [ 8939.299779] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8939.400453] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8939.406361] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8939.413464] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8939.419923] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8939.426564] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8939.437257] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8939.444979] Lustre: 280657:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8939.459193] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8939.588540] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8939.617042] Lustre: lustre-MDT0001: Aborting client recovery [ 8940.446486] loop2: detected capacity change from 0 to 204800000 [ 8940.628632] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8940.728314] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8940.755338] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8940.764496] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8940.788175] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8940.807656] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8940.823289] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8940.840270] Lustre: 280735:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8940.950602] Lustre: Found index 0 for lustre-OST0000, updating log [ 8940.981888] Lustre: 280754:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8941.070701] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8941.097890] Lustre: lustre-OST0000: Aborting recovery [ 8942.092312] loop3: detected capacity change from 0 to 204800000 [ 8942.273071] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8942.324337] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8942.334664] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8942.346472] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8942.353612] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8942.365450] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8942.372728] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8942.382873] Lustre: 280819:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8942.401382] Lustre: Found index 1 for lustre-OST0001, updating log [ 8942.426852] Lustre: 280825:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8942.472525] Lustre: lustre-OST0001: Aborting recovery [ 8947.726785] LustreError: 280600:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 8948.772692] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8948.774629] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8948.820668] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8948.837892] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8949.803431] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8949.807885] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8950.575672] LustreError: 280567:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 455us: rc = -2 [ 8951.122991] LustreError: 280568:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 575us: rc = -2 [ 8951.136633] LustreError: 280568:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8965.093340] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8965.094595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8965.094664] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8965.112266] Lustre: Skipped 3 previous similar messages [ 8965.120361] Lustre: Skipped 2 previous similar messages [ 8970.212190] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8970.229435] Lustre: Skipped 4 previous similar messages [ 8975.329858] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8975.340051] Lustre: Skipped 3 previous similar messages [ 8978.400204] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8978.641420] Lustre: server umount lustre-MDT0000 complete [ 8979.481637] LustreError: 280551:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446635 with bad export cookie 10225652145135204625 [ 8979.492364] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8980.449878] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8980.452464] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8980.463418] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8980.463730] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8980.491739] Lustre: Skipped 1 previous similar message [ 8984.484350] LustreError: 281050:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8984.489490] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8984.496046] LustreError: 281050:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8984.505107] Lustre: Skipped 2 previous similar messages [ 8985.957305] Lustre: server umount lustre-MDT0001 complete [ 8993.142788] Lustre: server umount lustre-OST0000 complete [ 9000.356980] Lustre: server umount lustre-OST0001 complete [ 9009.497995] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 10:44:25 (1788446665) [ 9039.964688] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 9050.250809] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 9089.469987] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9089.755677] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9089.783454] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9089.861375] Lustre: lustre-MDT0000: new disk, initializing [ 9090.003976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9090.012843] Lustre: Skipped 1 previous similar message [ 9090.033271] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9094.858921] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9105.011351] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9105.127820] Lustre: 284893:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9105.169170] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9105.173671] Lustre: Skipped 1 previous similar message [ 9105.226633] Lustre: lustre-MDT0001: new disk, initializing [ 9105.308254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9105.343568] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9105.359862] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9109.445506] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9119.379431] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9124.745507] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9131.134314] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9131.504481] Lustre: lustre-OST0000: new disk, initializing [ 9131.509170] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9131.514720] Lustre: 286862:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9131.605984] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9138.134743] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9138.281616] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9138.293518] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9138.515831] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9147.795817] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9152.832790] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9164.922733] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9176.983922] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9187.568909] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9225.698978] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9225.699933] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9225.701320] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9225.708018] LustreError: Skipped 1 previous similar message [ 9225.727616] Lustre: Skipped 1 previous similar message [ 9231.294957] Lustre: server umount lustre-OST0000 complete [ 9235.951201] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9241.412752] Lustre: server umount lustre-MDT0000 complete [ 9245.327736] LustreError: 284885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788446901 with bad export cookie 10225652145135207327 [ 9245.331704] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9245.339106] LustreError: 284885:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9245.841122] Lustre: server umount lustre-MDT0001 complete [ 9255.456863] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9260.512953] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9264.986814] Lustre: server umount MGS complete [ 9277.034107] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9277.439482] LustreError: 290879:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9277.468936] LustreError: 290892:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9277.543209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9282.400331] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9282.530424] LustreError: 290880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9287.654668] LustreError: 290879:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9291.201848] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9295.624935] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9303.569413] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9309.770460] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9316.990694] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9317.533632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9317.541113] Lustre: Skipped 1 previous similar message [ 9324.407956] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9331.990629] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9367.009771] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9367.018474] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9367.039423] Lustre: Skipped 1 previous similar message [ 9367.049684] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9367.066431] Lustre: Skipped 8 previous similar messages [ 9370.823155] Lustre: server umount lustre-OST0000 complete [ 9377.251710] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9381.155464] Lustre: server umount lustre-MDT0000 complete [ 9382.375785] LustreError: 290880:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9382.395040] LustreError: 290880:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9385.259787] LustreError: 291617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447041 with bad export cookie 10225652145135209266 [ 9385.264552] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9385.268868] LustreError: 291617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9385.646854] Lustre: server umount lustre-MDT0001 complete [ 9396.936316] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 9399.962696] Key type lgssc unregistered [ 9400.321855] LNet: 294822:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9400.329339] LNetError: 294822:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9401.393404] LNet: Removed LNI 192.168.202.133@tcp [ 9402.329338] Key type .llcrypt unregistered [ 9402.330875] Key type ._llcrypt unregistered [ 9441.908080] Key type ._llcrypt registered [ 9441.910811] Key type .llcrypt registered [ 9442.141636] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 9461.434954] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 9463.218277] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9463.322174] alg: No test for adler32 (adler32-zlib) [ 9464.517762] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 9464.840415] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 9466.480375] Key type lgssc registered [ 9467.607649] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9500.753554] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9500.776886] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9502.118780] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9502.140952] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9502.209538] Lustre: lustre-MDT0000: new disk, initializing [ 9502.361109] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9502.382124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9507.157925] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9516.548053] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9516.660852] Lustre: 298779:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9516.730903] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9516.734817] Lustre: Skipped 1 previous similar message [ 9516.793706] Lustre: lustre-MDT0001: new disk, initializing [ 9516.851875] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9516.875435] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9516.883038] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9522.292228] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9534.049723] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9541.018831] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9548.318824] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9548.854511] Lustre: lustre-OST0000: new disk, initializing [ 9548.873685] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9548.895412] Lustre: 300750:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9549.063780] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9554.587909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9554.606282] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9554.760233] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9556.670278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9566.889684] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9574.937545] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9587.390481] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9595.023458] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9600.850323] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9641.953408] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9641.955539] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9641.963944] LustreError: Skipped 1 previous similar message [ 9641.987636] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9641.999287] Lustre: Skipped 1 previous similar message [ 9645.044060] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9645.062542] Lustre: Skipped 1 previous similar message [ 9647.560893] Lustre: server umount lustre-OST0000 complete [ 9655.266256] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9655.267049] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9655.270780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9655.273201] Lustre: Skipped 2 previous similar messages [ 9657.366308] Lustre: server umount lustre-MDT0000 complete [ 9661.154268] LustreError: 298770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447317 with bad export cookie 12854902938326462226 [ 9661.159365] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9661.163900] LustreError: 298770:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9661.555676] Lustre: server umount lustre-MDT0001 complete [ 9670.017524] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9674.459262] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9678.600377] Lustre: server umount MGS complete [ 9689.353707] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9689.685093] LustreError: 304669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9689.703803] LustreError: 304669:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9689.717396] LustreError: 304682:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9689.769911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9693.630121] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9695.202603] LustreError: 304670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9699.301402] LustreError: 304669:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9703.278636] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9703.700771] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9709.045971] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9716.291921] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9721.537294] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9729.684672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9730.294258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9737.601461] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9746.539822] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9780.726565] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9780.758924] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9780.812140] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9780.834615] Lustre: Skipped 1 previous similar message [ 9784.801763] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9784.810700] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9784.846440] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9784.851764] Lustre: Skipped 1 previous similar message [ 9786.086074] Lustre: server umount lustre-OST0000 complete [ 9789.921651] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9789.930718] Lustre: Skipped 1 previous similar message [ 9795.044645] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9795.048331] Lustre: Skipped 3 previous similar messages [ 9804.256874] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9804.462204] Lustre: server umount lustre-MDT0000 complete [ 9805.285398] LustreError: 304670:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9805.312068] LustreError: 304670:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9808.238861] LustreError: 307397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788447464 with bad export cookie 12854902938326464172 [ 9808.240484] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9808.248817] LustreError: 307397:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 9808.505859] Lustre: server umount lustre-MDT0001 complete [ 9820.101849] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [ 9823.586530] Key type lgssc unregistered [ 9823.969754] LNet: 308617:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9823.978661] LNetError: 308617:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9823.995867] LNet: Removed LNI 192.168.202.133@tcp [ 9824.923329] Key type .llcrypt unregistered [ 9824.925518] Key type ._llcrypt unregistered [ 9847.252589] Key type ._llcrypt registered [ 9847.255316] Key type .llcrypt registered [ 9847.372237] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [ 9862.527333] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 9863.697164] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9863.843077] alg: No test for adler32 (adler32-zlib) [ 9864.898802] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [ 9865.153683] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [ 9866.816239] Key type lgssc registered [ 9868.154903] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9893.632807] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [ 9912.999087] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9913.060075] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9914.232597] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9914.246418] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9914.292196] Lustre: lustre-MDT0000: new disk, initializing [ 9914.392396] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9914.455484] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9919.654355] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [ 9930.293409] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9930.509445] Lustre: lustre-OST0000: new disk, initializing [ 9930.512596] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9930.516229] Lustre: Skipped 1 previous similar message [ 9930.520296] Lustre: 312667:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9930.581847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9935.950388] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9935.955153] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9935.980174] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9937.026494] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [10773.395061] Lustre: server umount lustre-MDT0000 complete [10777.742540] LustreError: 311853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448433 with bad export cookie 9776513870455194584 [10777.768673] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10788.022776] Lustre: server umount lustre-OST0000 complete [10824.739745] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [10838.286978] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [10875.975680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10876.337932] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10876.397194] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10876.512584] Lustre: lustre-MDT0000: new disk, initializing [10876.599927] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10876.620963] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10881.709420] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [10892.563893] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10892.751790] Lustre: 317013:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10892.840784] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10892.856487] Lustre: Skipped 1 previous similar message [10892.991849] Lustre: lustre-MDT0001: new disk, initializing [10893.149359] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10893.196458] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10893.210426] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10898.984358] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [10910.445214] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10917.250784] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10925.662037] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10926.071851] Lustre: lustre-OST0000: new disk, initializing [10926.077832] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10926.087689] Lustre: 318980:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10926.229923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10933.386796] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [10934.384605] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10934.411544] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10934.587025] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10943.164835] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10948.925426] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10949.234652] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10954.267348] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10954.622106] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10960.357277] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10960.359362] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10960.377578] LustreError: Skipped 1 previous similar message [10960.386715] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10960.440045] Lustre: Skipped 1 previous similar message [10965.497171] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10965.506737] Lustre: Skipped 2 previous similar messages [10970.594276] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10970.604713] Lustre: Skipped 1 previous similar message [10971.104961] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10971.320993] Lustre: server umount lustre-OST0000 complete [10975.207983] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10975.220477] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10975.244057] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10980.842723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10980.855135] Lustre: Skipped 2 previous similar messages [10980.929747] Lustre: server umount lustre-MDT0000 complete [10985.584484] LustreError: 317005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448641 with bad export cookie 9776513870458135403 [10985.603585] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10985.605039] LustreError: 317005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10986.287599] Lustre: server umount lustre-MDT0001 complete [10989.773611] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10991.855952] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 11:17:27 (1788448647) [11000.752903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11001.164545] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11006.356609] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11006.434883] LustreError: 321298:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11006.457242] LustreError: 321298:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11011.555553] LustreError: 321299:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11015.795089] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11016.083213] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11021.232579] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11029.179174] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11035.349486] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11042.618621] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11043.075135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11049.342804] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11057.598175] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11065.456445] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11065.674494] Lustre: lustre-OST0001: new disk, initializing [11065.677936] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11065.697926] Lustre: 324135:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11065.827608] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11071.635122] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11071.651316] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11071.684266] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11072.752164] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11072.757898] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11073.012326] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11073.836740] LustreError: 324430:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11073.857968] LustreError: 324430:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11075.875279] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11075.894824] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [11080.035737] LustreError: 324430:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11080.047128] LustreError: 324430:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11084.043737] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11084.270850] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11088.235683] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11088.244773] LustreError: 324434:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11090.078513] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff921450c82000.ost_server_uuid 50 [11091.600693] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff921450c82000.ost_server_uuid in FULL state after 0 sec [11097.343758] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff921450c82000.ost_server_uuid 50 [11098.919182] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff921450c82000.ost_server_uuid in FULL state after 0 sec [11100.635811] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11116.008057] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11116.011853] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11116.021901] LustreError: Skipped 1 previous similar message [11116.030406] Lustre: Skipped 1 previous similar message [11116.031868] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11116.065641] Lustre: Skipped 1 previous similar message [11129.312723] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11129.417932] Lustre: server umount lustre-OST0001 complete [11130.009922] LustreError: 323119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.202.33@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11133.411938] LustreError: 324640:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11133.440377] LustreError: 324640:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11138.531682] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11138.533021] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11138.534262] LustreError: 323119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11138.535965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11138.535974] Lustre: Skipped 8 previous similar messages [11138.541781] LustreError: Skipped 1 previous similar message [11138.582953] Lustre: Skipped 2 previous similar messages [11142.084697] Lustre: server umount lustre-OST0000 complete [11148.769666] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11148.796735] Lustre: Skipped 1 previous similar message [11152.530529] Lustre: server umount lustre-MDT0000 complete [11153.898587] LustreError: 322026:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11153.936748] LustreError: 322026:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11156.922791] LustreError: 321280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788448813 with bad export cookie 9776513870458136285 [11156.935597] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11156.939164] LustreError: 321280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11157.600956] Lustre: server umount lustre-MDT0001 complete [11169.719302] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [11173.254301] Key type lgssc unregistered [11173.516384] LNet: 326825:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11173.532398] LNetError: 326825:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11173.560389] LNet: Removed LNI 192.168.202.133@tcp [11174.499136] Key type .llcrypt unregistered [11174.503934] Key type ._llcrypt unregistered [11199.104715] Key type ._llcrypt registered [11199.107802] Key type .llcrypt registered [11199.193464] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [11200.571379] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11200.589265] alg: No test for adler32 (adler32-zlib) [11201.944226] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [11202.063081] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [11203.688170] Key type lgssc registered [11204.942876] Lustre: Echo OBD driver; http://www.lustre.org/ [11214.668385] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11214.693220] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11216.090372] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11220.572800] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11226.596235] LustreError: 328250:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11226.617841] LustreError: 328250:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11228.756420] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11231.729772] LustreError: 328249:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11235.784314] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11236.154606] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11240.857204] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11250.261707] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11250.833300] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11257.835285] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11261.043901] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11261.067984] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [11261.079690] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11265.607117] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11266.542979] LustreError: 330405:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11266.575568] LustreError: 330405:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11271.657419] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11271.684720] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11272.531570] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11272.774995] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11277.818921] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [11280.043609] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11287.155950] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11295.535886] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [11295.783924] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11400.673382] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11400.681314] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11400.693538] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11400.728479] Lustre: Skipped 1 previous similar message [11401.990571] Lustre: server umount lustre-OST0001 complete [11405.810038] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11405.833393] LustreError: 329929:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11409.904627] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11409.911506] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11409.925787] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11409.937643] Lustre: Skipped 2 previous similar messages [11416.034365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11416.037161] Lustre: Skipped 2 previous similar messages [11421.164480] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11421.174164] Lustre: Skipped 2 previous similar messages [11422.176367] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11422.347469] Lustre: server umount lustre-OST0000 complete [11425.761424] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11425.768411] LustreError: Skipped 1 previous similar message [11425.774991] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11425.792288] Lustre: Skipped 1 previous similar message [11425.797705] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11432.084455] Lustre: server umount lustre-MDT0000 complete [11435.996754] LustreError: 328230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449092 with bad export cookie 1951227082809956790 [11435.998385] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11436.003563] LustreError: 328230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11436.279500] Lustre: server umount lustre-MDT0001 complete [11448.729317] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [11451.138922] Key type lgssc unregistered [11451.394113] LNet: 333196:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11451.424618] LNetError: 333196:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11452.456364] LNet: Removed LNI 192.168.202.133@tcp [11453.612420] Key type .llcrypt unregistered [11453.614073] Key type ._llcrypt unregistered [11504.819909] Key type ._llcrypt registered [11504.828356] Key type .llcrypt registered [11504.963617] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [11506.866191] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11506.888744] alg: No test for adler32 (adler32-zlib) [11507.994425] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [11508.245480] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [11509.960739] Key type lgssc registered [11511.168857] Lustre: Echo OBD driver; http://www.lustre.org/ [11522.025562] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11522.045932] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11523.647719] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11528.373450] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11533.794733] LustreError: 335485:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11533.805616] LustreError: 335485:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11536.707820] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11537.176901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11542.439683] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11545.527041] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11552.618490] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11553.020430] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11559.997568] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11562.232241] LustreError: 336976:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11567.595223] LustreError: 336977:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11568.182176] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11568.480461] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11569.535340] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11569.558486] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11574.791498] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11585.501397] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11594.579237] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11610.603124] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11610.607246] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11610.620842] Lustre: Skipped 3 previous similar messages [11610.633363] Lustre: Skipped 2 previous similar messages [11615.713660] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11615.722328] Lustre: Skipped 4 previous similar messages [11615.952807] Lustre: server umount lustre-MDT0000 complete [11619.320754] LustreError: 335465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449275 with bad export cookie 899651411680341430 [11619.329983] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11619.333968] LustreError: 335465:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11620.017669] Lustre: server umount lustre-MDT0001 complete [11633.134378] Lustre: server umount lustre-OST0000 complete [11636.000169] Lustre: 334745:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788449276/real 1788449276] req@ffff8e0c36055880 x1875324870656768/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788449292 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11636.019428] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11637.766890] Lustre: server umount lustre-OST0001 complete [11657.736473] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [11667.978736] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11668.395717] LustreError: 340751:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11668.446581] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11672.556103] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11673.570909] LustreError: 340752:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11681.810982] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11682.335901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11687.942946] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11691.826232] Lustre: 341891:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11699.645319] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11700.151039] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11707.300173] LustreError: 342245:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11707.337216] LustreError: 342245:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11707.574303] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11713.545245] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11717.215591] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11722.747116] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11724.910452] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [11733.416144] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11737.017384] Lustre: 343761:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11743.205339] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11743.208333] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11743.258056] Lustre: Skipped 3 previous similar messages [11743.276889] Lustre: Skipped 3 previous similar messages [11747.439494] Lustre: server umount lustre-MDT0000 complete [11748.337297] LustreError: 341483:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11748.356984] LustreError: 341483:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [11751.674350] LustreError: 340731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449407 with bad export cookie 899651411680343131 [11751.678660] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11751.689358] LustreError: 340731:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11752.376182] Lustre: server umount lustre-MDT0001 complete [11767.792849] Lustre: server umount lustre-OST0000 complete [11769.827981] Lustre: 334746:0:(client.c:2504:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788449409/real 1788449409] req@ffff8e0b028f7b80 x1875324870707584/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788449425 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11769.862904] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11771.776238] Lustre: server umount lustre-OST0001 complete [11775.390576] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11777.029465] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11778.834904] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 11:30:34 (1788449434) [11805.804898] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [11817.313211] loop0: detected capacity change from 0 to 1099511627776 [11818.236766] SGI XFS with ACLs, security attributes, quota, no debug enabled [11818.243750] *** VALIDATE xfs *** [11818.256204] XFS (loop0): Mounting V5 Filesystem [11818.283293] XFS (loop0): Ending clean mount [11819.935232] loop1: detected capacity change from 0 to 18691697672192 [11955.173821] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [11967.313911] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [12007.333184] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12007.729610] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12007.769407] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12007.846816] Lustre: lustre-MDT0000: new disk, initializing [12007.914082] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12007.917354] Lustre: Skipped 1 previous similar message [12007.934218] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12012.594994] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12024.575238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12024.697278] Lustre: 348901:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12024.755286] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12024.768693] Lustre: Skipped 1 previous similar message [12024.910633] Lustre: lustre-MDT0001: new disk, initializing [12025.052444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12025.072757] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12030.625023] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12041.819252] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12048.804289] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12056.917601] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12057.220885] Lustre: lustre-OST0000: new disk, initializing [12057.225332] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12057.234266] Lustre: 350870:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12058.507541] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12058.517469] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12058.632182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12064.815826] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12075.007493] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12080.522120] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12080.875143] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12086.670793] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12086.883790] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12089.318616] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12089.321088] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12089.339291] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12089.364290] Lustre: Skipped 1 previous similar message [12094.437621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12094.443863] Lustre: Skipped 2 previous similar messages [12103.136155] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12103.289874] Lustre: server umount lustre-OST0000 complete [12107.247733] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12107.253879] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12107.262617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12107.267627] Lustre: Skipped 2 previous similar messages [12121.056735] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12121.460973] Lustre: server umount lustre-MDT0000 complete [12125.153903] LustreError: 350126:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12125.165759] LustreError: 350126:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12125.555192] LustreError: 348893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449781 with bad export cookie 899651411680344685 [12125.563978] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12125.570687] LustreError: 348893:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12126.028813] Lustre: server umount lustre-MDT0001 complete [12135.726570] XFS (loop0): Unmounting Filesystem [12140.959600] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 11:36:36 (1788449796) [12147.764392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12148.131235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12148.136359] Lustre: Skipped 2 previous similar messages [12152.604268] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12160.896197] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12166.113098] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12174.396567] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12180.138016] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12186.711251] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12194.188351] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12201.259828] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12216.811811] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12216.824098] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12216.841685] Lustre: Skipped 1 previous similar message [12216.843501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12216.843509] Lustre: Skipped 6 previous similar messages [12220.213544] Lustre: server umount lustre-OST0000 complete [12230.930383] Lustre: server umount lustre-MDT0000 complete [12232.167396] LustreError: 353428:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12232.192247] LustreError: 353428:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [12235.270321] LustreError: 353411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788449891 with bad export cookie 899651411680345567 [12235.285387] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12235.286798] LustreError: 353411:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12235.799628] Lustre: server umount lustre-MDT0001 complete [12248.663400] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [12252.285888] Key type lgssc unregistered [12252.599681] LNet: 357060:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12252.608420] LNetError: 357060:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12252.632189] LNet: Removed LNI 192.168.202.133@tcp [12253.641265] Key type .llcrypt unregistered [12253.643648] Key type ._llcrypt unregistered [12284.635500] Key type ._llcrypt registered [12284.638351] Key type .llcrypt registered [12284.856278] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [12286.218198] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12286.438180] alg: No test for adler32 (adler32-zlib) [12287.609835] Lustre: Lustre: Build Version: 2.17.58_2_gee80f9b [12287.899560] LNet: Added LNI 192.168.202.133@tcp [8/256/0/180] [12289.616218] Key type lgssc registered [12290.890950] Lustre: Echo OBD driver; http://www.lustre.org/ [12299.817288] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12299.839041] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12301.335373] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12306.300493] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12311.537162] LustreError: 358627:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12311.572391] LustreError: 358627:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12315.397062] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12315.821992] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12320.478577] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12328.441500] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12334.379326] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12341.748818] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12342.273263] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12349.536635] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12358.589662] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12366.519774] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [12366.848945] Lustre: server umount lustre-MDT0001 complete [12367.335227] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12367.351545] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12367.384976] LustreError: 358628:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12370.922251] LustreError: 358627:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12370.925629] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12376.035737] LustreError: 358622:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12376.062053] LustreError: 358622:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12378.189731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12378.329486] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12378.336936] Lustre: Found index 1 for lustre-MDT0001, updating log [12378.373026] Lustre: 358615:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12378.632941] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12383.036899] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12383.736781] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12385.279928] LustreError: 362197:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12385.289245] LustreError: 362197:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12385.300691] LustreError: 358621:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12385.368886] LustreError: 362203:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -17 [12385.380792] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12385.380792] [12392.140537] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [12393.957282] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12393.958074] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12393.980196] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12397.205851] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [12397.225948] Lustre: Skipped 1 previous similar message [12398.017408] Lustre: server umount lustre-MDT0001 complete [12399.079742] LustreError: 361818:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12399.116240] LustreError: 361818:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12406.997391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12407.099341] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12407.103426] Lustre: Found index 1 for lustre-MDT0001, updating log [12407.156140] Lustre: 358617:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12407.413296] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12412.769306] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12412.908966] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12416.007705] LustreError: 363300:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12416.016293] LustreError: 363300:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12416.029804] LustreError: 358621:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12416.042550] LustreError: 358621:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12416.125601] LustreError: 363303:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12416.132068] LustreError: 363303:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12416.136379] LustreError: 363303:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -17 [12416.146303] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.202.133@tcp [12416.146303] [12719.783220] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (stopping) [12720.103229] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12720.115556] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12720.127518] Lustre: Skipped 2 previous similar messages [12724.888472] LustreError: 358622:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.202.33@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12724.905890] LustreError: 358622:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [12724.956204] Lustre: server umount lustre-MDT0001 complete [12734.828773] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12734.921417] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12734.927280] Lustre: Found index 1 for lustre-MDT0001, updating log [12734.980052] Lustre: 358616:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12735.120686] Lustre: lustre-MDT0001: Not available for connect from 192.168.202.33@tcp (not set up) [12735.127201] Lustre: Skipped 2 previous similar messages [12735.316333] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12739.633490] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [12740.583275] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12741.119619] LustreError: 372170:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12741.132124] LustreError: 372170:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12741.146855] LustreError: 358621:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12741.156259] LustreError: 358621:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12741.227597] LustreError: 372177:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12741.251796] LustreError: 372177:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.202.133@tcp: cfg command failed: rc = -114 [12741.263716] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12741.263716] [13063.169412] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13063.187754] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13063.190295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13063.214051] Lustre: Skipped 2 previous similar messages [13068.874842] Lustre: server umount lustre-MDT0000 complete [13073.008398] LustreError: 358610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450729 with bad export cookie 17913895153858739942 [13073.021378] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13073.024215] LustreError: 358610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13073.385832] LustreError: 359643:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13073.391713] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13073.437710] LustreError: 359643:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [13079.477836] Lustre: server umount lustre-MDT0001 complete [13083.721429] Lustre: server umount lustre-OST0000 complete [13095.254133] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [13107.508480] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [13143.349725] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 11:53:19 (1788450799) [13150.801790] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13151.208512] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13151.287153] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13151.472650] Lustre: lustre-MDT0000: new disk, initializing [13151.684706] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13151.726828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13157.974767] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [13172.384916] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13172.549986] Lustre: 382907:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13172.601410] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13172.609266] Lustre: Skipped 1 previous similar message [13172.665803] Lustre: lustre-MDT0001: new disk, initializing [13172.750283] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13172.796494] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13172.809640] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13177.127222] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [13187.290359] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13192.593592] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13198.417156] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13198.677305] Lustre: lustre-OST0000: new disk, initializing [13198.681335] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13198.687905] Lustre: 384876:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13198.758481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13204.546536] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13204.559510] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13204.665120] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13205.060075] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_default_debug -1 all [13214.105218] Lustre: DEBUG MARKER: oleg233-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13222.888755] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [13222.892161] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13222.912028] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13222.916819] Lustre: Skipped 7 previous similar messages [13224.174973] Lustre: server umount lustre-MDT0000 complete [13225.447231] LustreError: 384148:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13225.466350] LustreError: 384148:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [13227.564156] LustreError: 382899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788450883 with bad export cookie 17913895153858741895 [13227.573926] LustreError: 382899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [13227.575679] LustreError: MGC192.168.202.133@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13227.592806] LustreError: 383714:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [13228.090924] Lustre: server umount lustre-MDT0001 complete [13241.863469] Lustre: server umount lustre-OST0000 complete [13271.119837] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing set_hostid [13281.875470] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing load_modules_local [13314.172380] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12979 sec ========= 11:56:09 (1788450969) [13316.387762] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 11:56:11 (1788450971) === [13320.707726] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 11:56:16 (1788450976) === [13367.998489] Lustre: DEBUG MARKER: oleg233-server.virtnet: executing unload_modules_local [13371.433665] Key type lgssc unregistered [13371.847599] LNet: 391242:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13371.855951] LNetError: 391242:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13371.880201] LNet: Removed LNI 192.168.202.133@tcp [13373.024045] Key type .llcrypt unregistered [13373.027908] Key type ._llcrypt unregistered