[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 1469460128 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003290] x2apic enabled [ 0.004006] Switched APIC routing to physical x2apic. [ 0.005011] kvm-guest: setup PV IPIs [ 0.008456] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.009017] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.010008] pid_max: default: 32768 minimum: 301 [ 0.011111] LSM: Security Framework initializing [ 0.012033] Yama: becoming mindful. [ 0.013024] SELinux: Initializing. [ 0.014045] *** VALIDATE selinux *** [ 0.022592] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026645] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027184] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028077] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029081] *** VALIDATE tmpfs *** [ 0.030448] *** VALIDATE proc *** [ 0.031245] *** VALIDATE cgroup *** [ 0.032007] *** VALIDATE cgroup2 *** [ 0.034049] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.035126] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038021] Spectre V2 : User space: Vulnerable [ 0.039004] Speculative Store Bypass: Vulnerable [ 0.042396] debug: unmapping init [mem 0xffffffff8d259000-0xffffffff8d260fff] [ 0.044943] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045582] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046015] ... version: 2 [ 0.047011] ... bit width: 48 [ 0.048011] ... generic registers: 4 [ 0.049009] ... value mask: 0000ffffffffffff [ 0.050016] ... max period: 00007fffffffffff [ 0.051014] ... fixed-purpose events: 3 [ 0.052011] ... event mask: 000000070000000f [ 0.053265] rcu: Hierarchical SRCU implementation. [ 0.056742] smp: Bringing up secondary CPUs ... [ 0.058028] x86: Booting SMP configuration: [ 0.059019] .... node #0, CPUs: #1 #2 #3 [ 0.096148] smp: Brought up 1 node, 4 CPUs [ 0.098011] smpboot: Max logical packages: 1 [ 0.099011] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.157216] node 0 deferred pages initialised in 56ms [ 0.164285] devtmpfs: initialized [ 0.165255] x86/mm: Memory block size: 128MB [ 0.169630] gcov: version magic: 0x41383552 [ 0.174542] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.178071] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.185315] pinctrl core: initialized pinctrl subsystem [ 0.187169] [ 0.188006] ************************************************************* [ 0.192011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.195013] ** ** [ 0.199013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.202030] ** ** [ 0.205039] ** This means that this kernel is built to expose internal ** [ 0.208010] ** IOMMU data structures, which may compromise security on ** [ 0.212014] ** your system. ** [ 0.215021] ** ** [ 0.219009] ** If you see this message and you are not debugging the ** [ 0.222009] ** kernel, report this immediately to your vendor! ** [ 0.226009] ** ** [ 0.229037] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.233042] ************************************************************* [ 0.237553] NET: Registered protocol family 16 [ 0.239381] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.242060] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.246041] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.250027] cpuidle: using governor menu [ 0.253011] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.255503] PCI: Using configuration type 1 for base access [ 0.258112] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.267063] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.268018] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.269350] cryptd: max_cpu_qlen set to 1000 [ 0.271487] ACPI: Added _OSI(Module Device) [ 0.272000] ACPI: Added _OSI(Processor Device) [ 0.272000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.275010] ACPI: Added _OSI(Processor Aggregator Device) [ 0.282282] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.290516] ACPI: Interpreter enabled [ 0.293036] ACPI: PM: (supports S0 S3 S4 S5) [ 0.294009] ACPI: Using IOAPIC for interrupt routing [ 0.297083] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.301530] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.318596] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.321046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.325019] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.330070] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.336750] acpiphp: Slot [2] registered [ 0.338122] acpiphp: Slot [5] registered [ 0.340148] acpiphp: Slot [6] registered [ 0.342104] acpiphp: Slot [7] registered [ 0.345157] acpiphp: Slot [8] registered [ 0.347100] acpiphp: Slot [9] registered [ 0.348100] acpiphp: Slot [10] registered [ 0.350087] acpiphp: Slot [3] registered [ 0.352096] acpiphp: Slot [4] registered [ 0.354130] acpiphp: Slot [11] registered [ 0.355228] acpiphp: Slot [12] registered [ 0.357075] acpiphp: Slot [13] registered [ 0.359069] acpiphp: Slot [14] registered [ 0.360072] acpiphp: Slot [15] registered [ 0.361066] acpiphp: Slot [16] registered [ 0.363063] acpiphp: Slot [17] registered [ 0.364086] acpiphp: Slot [18] registered [ 0.366070] acpiphp: Slot [19] registered [ 0.368079] acpiphp: Slot [20] registered [ 0.370119] acpiphp: Slot [21] registered [ 0.371139] acpiphp: Slot [22] registered [ 0.373064] acpiphp: Slot [23] registered [ 0.375070] acpiphp: Slot [24] registered [ 0.377079] acpiphp: Slot [25] registered [ 0.378136] acpiphp: Slot [26] registered [ 0.380064] acpiphp: Slot [27] registered [ 0.382063] acpiphp: Slot [28] registered [ 0.384077] acpiphp: Slot [29] registered [ 0.385061] acpiphp: Slot [30] registered [ 0.387091] acpiphp: Slot [31] registered [ 0.389046] PCI host bridge to bus 0000:00 [ 0.390011] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.393011] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.396069] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.399017] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.402015] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.405017] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.408155] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.411066] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.415270] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.423011] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.431106] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.434014] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.436014] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.439011] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.442516] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.445724] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.449086] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.452817] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.458010] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.472010] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.477010] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.483078] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.506015] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.687011] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.745012] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.763279] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.777012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.791012] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.831012] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.857199] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.884013] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.984015] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 1.004013] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 1.022396] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 1.036013] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 1.043011] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 1.057013] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 1.067474] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 1.073013] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 1.081013] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 1.095014] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 1.107416] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 1.114015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 1.121017] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 1.139015] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 1.151798] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 1.154387] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 1.156311] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 1.158337] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 1.161277] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 1.165505] iommu: Default domain type: Passthrough [ 1.168368] SCSI subsystem initialized [ 1.169202] ACPI: bus type USB registered [ 1.171073] usbcore: registered new interface driver usbfs [ 1.173071] usbcore: registered new interface driver hub [ 1.175111] usbcore: registered new device driver usb [ 1.178142] pps_core: LinuxPPS API ver. 1 registered [ 1.180011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 1.183052] PTP clock support registered [ 1.185249] EDAC MC: Ver: 3.0.0 [ 1.192112] PCI: Using ACPI for IRQ routing [ 1.200103] NetLabel: Initializing [ 1.205011] NetLabel: domain hash size = 128 [ 1.209009] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 1.229119] NetLabel: unlabeled traffic allowed by default [ 1.232118] vgaarb: loaded [ 1.235034] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 1.380032] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 1.388000] clocksource: Switched to clocksource kvm-clock [ 1.534103] VFS: Disk quotas dquot_6.6.0 [ 1.536029] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.545981] *** VALIDATE ramfs *** [ 1.550948] *** VALIDATE hugetlbfs *** [ 1.559183] pnp: PnP ACPI init [ 1.566610] pnp: PnP ACPI: found 6 devices [ 1.665268] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.667819] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.674020] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.681942] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.690581] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.699114] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.720113] NET: Registered protocol family 2 [ 1.729861] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.755614] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.776612] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.804647] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.820512] TCP: Hash tables configured (established 65536 bind 65536) [ 1.832505] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.843171] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.853500] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.863450] NET: Registered protocol family 1 [ 1.874461] RPC: Registered named UNIX socket transport module. [ 1.882199] RPC: Registered udp transport module. [ 1.887455] RPC: Registered tcp transport module. [ 1.892146] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.897754] NET: Registered protocol family 44 [ 1.903535] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.907584] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.910932] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.913958] PCI: CLS 0 bytes, default 64 [ 1.915729] Unpacking initramfs... [ 3.991083] debug: unmapping init [mem 0xffff993a7cc54000-0xffff993a7ffbffff] [ 3.995312] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 3.997292] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 4.000296] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 4.719059] Initialise system trusted keyrings [ 4.721059] Key type blacklist registered [ 4.722947] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 4.734347] zbud: loaded [ 4.737911] *** VALIDATE nfs *** [ 4.739045] *** VALIDATE nfs4 *** [ 4.740816] pstore: using deflate compression [ 4.744828] Platform Keyring initialized [ 4.846626] NET: Registered protocol family 38 [ 4.848748] Key type asymmetric registered [ 4.850748] Asymmetric key parser 'x509' registered [ 4.852913] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 4.856685] io scheduler mq-deadline registered [ 4.858657] io scheduler kyber registered [ 4.860703] io scheduler bfq registered [ 4.862744] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 4.866664] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 4.870380] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 4.873749] ACPI: Power Button [PWRF] [ 4.881917] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 4.889320] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 4.912113] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 4.922994] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 4.967148] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 4.998293] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 5.029355] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 5.036531] Non-volatile memory driver v1.3 [ 5.038991] Linux agpgart interface v0.103 [ 5.069067] virtio_blk virtio1: [vda] 145904 512-byte logical blocks (74.7 MB/71.2 MiB) [ 5.072190] vda: detected capacity change from 0 to 74702848 [ 5.220713] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 5.223853] vdb: detected capacity change from 0 to 1073741824 [ 5.252482] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.256304] vdc: detected capacity change from 0 to 2621440000 [ 5.293291] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 5.297419] vdd: detected capacity change from 0 to 2621440000 [ 5.312252] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.315420] vde: detected capacity change from 0 to 4294967296 [ 5.333818] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 5.337334] vdf: detected capacity change from 0 to 4294967296 [ 5.347056] libphy: Fixed MDIO Bus: probed [ 5.352082] usbcore: registered new interface driver usbserial_generic [ 5.355129] usbserial: USB Serial support registered for generic [ 5.357982] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 5.363497] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 5.365784] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 5.368724] mousedev: PS/2 mouse device common for all mice [ 5.373397] rtc_cmos 00:05: RTC can wake from S4 [ 5.377691] rtc_cmos 00:05: registered as rtc0 [ 5.379474] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 5.381744] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 5.385015] intel_pstate: CPU model not supported [ 5.480750] hid: raw HID events driver (C) Jiri Kosina [ 5.489171] usbcore: registered new interface driver usbhid [ 5.490578] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 5.494759] usbhid: USB HID core driver [ 5.494920] drop_monitor: Initializing network drop monitor service [ 5.495052] Initializing XFRM netlink socket [ 5.495437] NET: Registered protocol family 10 [ 5.496945] Segment Routing with IPv6 [ 5.505691] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 5.508700] NET: Registered protocol family 17 [ 5.512608] mpls_gso: MPLS GSO support [ 5.523347] RAS: Correctable Errors collector initialized. [ 5.525446] AVX version of gcm_enc/dec engaged. [ 5.527112] AES CTR mode by8 optimization enabled [ 5.602698] sched_clock: Marking stable (5602627195, 0)->(7196810412, -1594183217) [ 5.607549] registered taskstats version 1 [ 5.609679] Loading compiled-in X.509 certificates [ 5.611946] zswap: loaded using pool lzo/zbud [ 5.636671] Key type big_key registered [ 5.649609] Key type encrypted registered [ 5.650962] ima: No TPM chip found, activating TPM-bypass! [ 5.652897] ima: Allocated hash algorithm: sha1 [ 5.654795] ima: No architecture policies found [ 5.656603] evm: Initialising EVM extended attributes: [ 5.658618] evm: security.selinux [ 5.659789] evm: security.ima [ 5.660885] evm: security.capability [ 5.662367] evm: HMAC attrs: 0x1 [ 5.664761] rtc_cmos 00:05: setting system clock to 2026-07-30 05:11:23 UTC (1785388283) [ 5.672558] debug: unmapping init [mem 0xffffffff8e203000-0xffffffff8e3fffff] [ 5.675851] debug: unmapping init [mem 0xffffffff8cf82000-0xffffffff8d258fff] [ 5.687195] Write protecting the kernel read-only data: 28672k [ 5.690928] debug: unmapping init [mem 0xffffffff8b603000-0xffffffff8b7fffff] [ 5.693926] debug: unmapping init [mem 0xffffffff8bf14000-0xffffffff8bffffff] [ 5.728859] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 5.738635] systemd[1]: Detected virtualization kvm. [ 5.740907] systemd[1]: Detected architecture x86-64. [ 5.743312] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 5.769893] systemd[1]: No hostname configured. [ 5.771713] systemd[1]: Set hostname to . [ 5.773920] random: systemd: uninitialized urandom read (16 bytes read) [ 5.776666] systemd[1]: Initializing machine ID from random generator. [ 6.009663] random: systemd: uninitialized urandom read (16 bytes read) [ 6.012725] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 6.023569] random: systemd: uninitialized urandom read (16 bytes read) [ 6.026062] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 6.031873] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Local File Systems. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Listening on Journal Socket. Starting Create Volatile Files and Directories... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Reached target Sockets. Starting Journal Service... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 7.562318] device-mapper: uevent: version 1.0.3 [ 7.564393] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK [0[ 8.971340] random: fast init done m] Started udev Coldplug all Devices. [ 9.088105] virtio_net virtio0 ens2: renamed from eth0 [ 9.538686] scsi host0: ata_piix [ 9.590244] scsi host1: ata_piix [ 9.591695] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 9.594032] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 15.256834] random: crng init done [ 15.260802] random: 7 urandom warning(s) missed due to ratelimiting [ 18.007701] dracut-initqueue[569]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 19.706406] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Swap. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 22.656771] printk: systemd: 26 output lines suppressed due to ratelimiting [ 23.336483] SELinux: Disabled at runtime. [ 23.404563] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 23.413895] systemd[1]: Detected virtualization kvm. [ 23.415966] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 26.642625] systemd[1]: initrd-switch-root.service: Succeeded. [ 26.649323] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 26.670277] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 26.689794] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 26.702351] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 26.741730] systemd[1]: Starting Journal Service... Starting Journal Service... [ 26.770569] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice User and Session Slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Mounting Huge Pages File System... Mounting POSIX Message Queue File System... [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target rpc_pipefs.target. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Slices. [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Apply Kernel Variables... [ 27.574582] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. [ OK ] Stopped target Initrd File Systems. Mounting Kernel Debug File System... [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Journal Service. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted Kernel Debug File System. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 28.868186] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 30.189433] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 30.254009] hrtimer: interrupt took 14001036 ns [ 30.386137] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 31.106660] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 31.277033] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (9s / no limit) [*** ] A start job is running for Configur…-only root support (9s / no limit)[ 36.522703] Key type dns_resolver registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ *** ] A start job is running for Configur…only root support (10s / no limit) [ ***] A start job is running for Configur…only root support (11s / no limit) [ **] A start job is running for Configur…only root support (12s / no limit)[ 38.983799] NFS: Registering the id_resolver key type [ 39.034261] Key type id_resolver registered [ 39.035870] Key type id_legacy registered [ *] A start job is running for Configur…only root support (12s / no limit) [ **] A start job is running for Configur…only root support (13s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Create Volatile Files and Directories... Starting Load/Save Random Seed... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Login Service... Starting Restore /run/initramfs on shutdown... [ OK ] Started dnf makecache --timer. [ OK ] Started irqbalance daemon. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Login Service. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg319-server login: [ 141.876474] libcfs: loading out-of-tree module taints kernel. [ 141.981472] Key type ._llcrypt registered [ 141.987754] Key type .llcrypt registered [ 142.130951] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 171.571706] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 175.411596] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 175.447110] alg: No test for adler32 (adler32-zlib) [ 178.474977] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 181.454403] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 183.503162] Key type lgssc registered [ 188.034378] Lustre: Echo OBD driver; http://www.lustre.org/ [ 212.652121] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 276.244375] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 295.179212] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 295.258444] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 296.688760] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 296.735219] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 296.977635] Lustre: lustre-MDT0000: new disk, initializing [ 297.086356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 297.100605] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 305.161591] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 330.048588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 330.181362] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 330.245327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 330.252097] Lustre: Skipped 1 previous similar message [ 330.351244] Lustre: lustre-MDT0001: new disk, initializing [ 330.423730] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 330.454402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 330.461392] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 336.200974] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 343.522677] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 358.253141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 358.934542] Lustre: lustre-OST0000: new disk, initializing [ 358.952865] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 358.977892] Lustre: 8457:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 359.134360] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 367.162509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 367.175276] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 367.359423] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 369.605247] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 392.956551] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 393.296209] Lustre: lustre-OST0001: new disk, initializing [ 393.310289] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 393.329200] Lustre: 9533:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 393.476516] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 401.979909] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 402.005127] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 402.175710] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 405.136953] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 423.336521] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 430.255774] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 441.100025] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing check_logdir /tmp/testlogs/ [ 450.493327] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing yml_node [ 459.008896] Lustre: DEBUG MARKER: Client: 2.17.56.2 [ 463.615585] Lustre: DEBUG MARKER: MDS: 2.17.56.2 [ 468.155967] Lustre: DEBUG MARKER: OSS: 2.17.56.2 [ 471.208487] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jul 30 01:19:06 EDT 2026 [ 501.408606] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 504.149779] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 525.292225] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 525.296094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 525.302269] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 525.313675] Lustre: Skipped 3 previous similar messages [ 525.351878] Lustre: Skipped 3 previous similar messages [ 529.947870] Lustre: server umount lustre-MDT0000 complete [ 535.331944] LustreError: 6508:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785388813 with bad export cookie 12565155927879628326 [ 535.338483] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 535.347630] LustreError: 6508:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 535.520075] LustreError: 6523:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 535.524346] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.554431] LustreError: 6523:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 535.603156] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 535.612212] Lustre: Skipped 1 previous similar message [ 540.640714] LustreError: 6524:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 540.641068] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 540.706726] Lustre: Skipped 1 previous similar message [ 542.238985] Lustre: server umount lustre-MDT0001 complete [ 548.875030] Lustre: server umount lustre-OST0000 complete [ 555.115290] Lustre: server umount lustre-OST0001 complete [ 564.975748] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 576.508355] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 630.372802] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 630.938428] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 631.062578] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 631.228882] Lustre: lustre-MDT0000: new disk, initializing [ 631.370433] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 631.409721] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 639.250712] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 656.420468] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 656.690731] Lustre: 15093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 656.894251] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 656.910040] Lustre: Skipped 1 previous similar message [ 657.131240] Lustre: lustre-MDT0001: new disk, initializing [ 657.380755] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 657.444198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 657.464152] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 664.340738] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 677.808933] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 691.584432] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 701.388866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 702.142332] Lustre: lustre-OST0000: new disk, initializing [ 702.158366] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 702.174671] Lustre: 17071:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 702.492274] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 703.477950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 703.493452] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 703.708897] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 712.831069] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 727.171355] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 735.883480] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 736.375860] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 742.995403] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 743.253734] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 749.536753] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 749.547139] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 749.561774] Lustre: Skipped 1 previous similar message [ 749.570323] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 755.169971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 755.179566] Lustre: Skipped 3 previous similar messages [ 760.287160] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 760.946762] Lustre: server umount lustre-OST0000 complete [ 770.016604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 770.030960] LustreError: Skipped 1 previous similar message [ 770.038658] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 770.061417] Lustre: Skipped 1 previous similar message [ 770.082286] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 770.088664] Lustre: Skipped 2 previous similar messages [ 773.236962] Lustre: server umount lustre-MDT0000 complete [ 775.651713] LustreError: 15105:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 775.683410] LustreError: 15105:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 778.523422] LustreError: 15892:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389056 with bad export cookie 12565155927879629866 [ 778.531454] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 778.543519] LustreError: 15892:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 779.152512] Lustre: server umount lustre-MDT0001 complete [ 785.898183] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 01:24:22 (1785389062) [ 797.853120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 799.036222] LustreError: 19340:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 799.266649] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 804.323148] LustreError: 19341:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 808.037905] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 813.543747] LustreError: 19340:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 813.573231] LustreError: 19340:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 819.993993] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 827.339727] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 838.948436] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 846.928517] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 855.403650] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 864.305595] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 873.776425] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 877.951421] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 887.284478] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 887.304311] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 887.327823] Lustre: Skipped 1 previous similar message [ 887.338250] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 887.344697] Lustre: Skipped 1 previous similar message [ 892.915668] Lustre: server umount lustre-MDT0000 complete [ 893.925483] LustreError: 20584:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 893.946619] LustreError: 20584:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 897.694054] LustreError: 19322:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389175 with bad export cookie 12565155927879630748 [ 897.716584] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 898.344316] Lustre: server umount lustre-MDT0001 complete [ 915.556644] Lustre: server umount lustre-OST0000 complete [ 945.798840] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 962.863748] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 963.494183] LustreError: 23730:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 963.619215] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 963.626617] Lustre: Skipped 2 previous similar messages [ 970.073415] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 982.236561] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 987.711251] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 992.237261] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 1005.492620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1018.010881] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1032.934654] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1033.119557] Lustre: lustre-OST0001: new disk, initializing [ 1033.124627] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1033.134224] Lustre: 26051:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1034.947357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1034.965422] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1035.145944] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1045.639242] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1059.803653] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1064.691220] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 1084.897120] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1084.919055] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1084.955514] Lustre: Skipped 2 previous similar messages [ 1084.966434] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1084.989068] Lustre: Skipped 3 previous similar messages [ 1089.786327] Lustre: server umount lustre-MDT0000 complete [ 1091.556665] LustreError: 23725:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1091.588930] LustreError: 23725:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 1096.949531] LustreError: 23711:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389374 with bad export cookie 12565155927879631595 [ 1096.971137] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1096.984762] LustreError: 23711:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1097.663111] Lustre: server umount lustre-MDT0001 complete [ 1115.475601] Lustre: server umount lustre-OST0000 complete [ 1118.187670] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785389379/real 1785389379] req@ffff993ac4675c00 x1872115489085312/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785389395 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1118.233484] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1118.260246] Lustre: Skipped 3 previous similar messages [ 1121.957734] Lustre: server umount lustre-OST0001 complete [ 1144.941300] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 1162.105386] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1162.822557] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1162.829690] Lustre: Skipped 3 previous similar messages [ 1171.451957] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1183.073394] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1190.643622] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1194.535977] Lustre: 30227:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1204.712903] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1214.803087] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1223.653718] LustreError: 30582:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1223.679236] LustreError: 30582:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [ 1226.586865] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1234.590351] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1244.202285] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1248.948261] Lustre: 32107:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1260.514160] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1260.528680] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1260.549179] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1260.558515] Lustre: Skipped 4 previous similar messages [ 1264.720458] Lustre: server umount lustre-MDT0000 complete [ 1269.607130] LustreError: 29058:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389547 with bad export cookie 12565155927879632890 [ 1269.625220] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1269.971556] Lustre: server umount lustre-MDT0001 complete [ 1284.580923] Lustre: server umount lustre-OST0000 complete [ 1289.760137] Lustre: 3643:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785389551/real 1785389551] req@ffff9939c797c380 x1872115489144704/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785389567 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1290.561510] Lustre: server umount lustre-OST0001 complete [ 1302.567273] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 01:32:59 (1785389579) [ 1340.092662] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 1360.059873] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1367.180868] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1378.559509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1385.015902] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1388.310839] Lustre: 36061:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1396.987223] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1408.170901] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1422.669672] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1423.154852] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 1423.157933] Lustre: Skipped 6 previous similar messages [ 1431.646791] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1442.626611] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1448.091740] Lustre: 37941:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1459.171642] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1459.177096] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1459.186327] Lustre: Skipped 6 previous similar messages [ 1459.202978] Lustre: Skipped 7 previous similar messages [ 1463.061119] Lustre: server umount lustre-MDT0000 complete [ 1469.787367] LustreError: 34894:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389747 with bad export cookie 12565155927879633996 [ 1469.792145] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1469.812293] LustreError: 34894:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 1470.424353] Lustre: server umount lustre-MDT0001 complete [ 1486.641792] Lustre: server umount lustre-OST0000 complete [ 1490.897397] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785389752/real 1785389752] req@ffff9939c78dd880 x1872115489206656/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785389768 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1492.657967] Lustre: server umount lustre-OST0001 complete [ 1504.708001] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 01:36:21 (1785389781) [ 1528.666058] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 1544.705939] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1545.454377] LustreError: 40111:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1545.483606] LustreError: 40111:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 25 previous similar messages [ 1551.496270] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1562.659928] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1570.139306] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1573.953356] Lustre: 41261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1585.897119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1596.735454] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1608.001461] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1616.168723] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1627.487569] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1633.457254] Lustre: 43139:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1634.820952] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1634.825826] Lustre: Skipped 1 previous similar message [ 1639.395810] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1639.437827] Lustre: Skipped 4 previous similar messages [ 1643.746067] Lustre: server umount lustre-MDT0000 complete [ 1649.380690] LustreError: 40093:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785389927 with bad export cookie 12565155927879635102 [ 1649.382194] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1649.410445] LustreError: 40093:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1656.377401] Lustre: server umount lustre-MDT0001 complete [ 1668.637016] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1674.208766] LustreError: 3642:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff993aff97aa00 x1872115489266816/t0(0) o250->MGC192.168.203.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1682.309527] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1684.340401] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1695.167686] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1701.370501] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1701.386786] Lustre: Skipped 1 previous similar message [ 1702.888792] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1713.885324] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1715.991595] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1727.004595] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1730.436870] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1746.971197] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1746.977158] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1747.005034] Lustre: Skipped 11 previous similar messages [ 1750.113730] Lustre: server umount lustre-MDT0000 complete [ 1756.316201] LustreError: 40091:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785390034 with bad export cookie 12565155927879636229 [ 1756.317919] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1756.338932] LustreError: 40091:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1757.828839] Lustre: server umount lustre-MDT0001 complete [ 1774.560121] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785390036/real 1785390036] req@ffff993aef264380 x1872115489313024/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785390052 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1775.776095] Lustre: server umount lustre-OST0000 complete [ 1779.747691] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785390041/real 1785390041] req@ffff993ac4f64e00 x1872115489313536/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785390057 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1779.787365] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 1782.640939] Lustre: server umount lustre-OST0001 complete [ 1795.144756] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 01:41:11 (1785390071) [ 1815.197120] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 1831.126211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1837.810713] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1849.793658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1856.899789] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1862.024827] Lustre: 48912:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1872.035330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1881.664981] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1893.254677] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1903.479444] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 1913.455985] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1919.877565] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1942.497233] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1942.504895] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1942.518879] Lustre: Skipped 8 previous similar messages [ 1948.896087] Lustre: server umount lustre-MDT0000 complete [ 1954.626674] LustreError: 50797:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785390232 with bad export cookie 12565155927879637020 [ 1954.635745] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1954.641225] LustreError: 50797:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1955.424946] Lustre: server umount lustre-MDT0001 complete [ 1971.900388] Lustre: server umount lustre-OST0000 complete [ 1974.752073] Lustre: 3644:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785390236/real 1785390236] req@ffff993affaa5880 x1872115489384448/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785390252 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1976.997993] Lustre: server umount lustre-OST0001 complete [ 1989.807944] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 01:44:26 (1785390266) [ 1992.726349] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1995.048130] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 01:44:31 (1785390271) [ 2009.843319] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 2020.866739] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2053.757942] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2054.327131] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2054.342560] Lustre: Skipped 1 previous similar message [ 2054.448419] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2054.608537] Lustre: lustre-MDT0000: new disk, initializing [ 2054.689915] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2054.702591] Lustre: Skipped 10 previous similar messages [ 2054.735242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2060.759892] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2074.710874] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2085.251139] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2085.767309] Lustre: lustre-OST0000: new disk, initializing [ 2085.779256] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2085.791071] Lustre: Skipped 1 previous similar message [ 2085.795912] Lustre: 55016:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2087.143943] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2087.151827] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2087.179888] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2096.952990] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2110.678767] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2114.296298] Lustre: DEBUG MARKER: create test files [ 2164.202914] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2167.034525] Lustre: server umount lustre-OST0000 complete [ 2174.326328] Lustre: server umount lustre-MDT0000 complete [ 2190.774296] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 2194.528176] Key type lgssc unregistered [ 2194.876797] LNet: 56656:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2194.900283] LNetError: 56656:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2194.924661] LNet: Removed LNI 192.168.203.119@tcp [ 2196.377132] Key type .llcrypt unregistered [ 2196.379800] Key type ._llcrypt unregistered [ 2232.456846] Key type ._llcrypt registered [ 2232.471701] Key type .llcrypt registered [ 2232.744315] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2234.828968] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2234.855994] alg: No test for adler32 (adler32-zlib) [ 2236.242738] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 2236.723571] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 2238.455149] Key type lgssc registered [ 2240.087311] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2250.867769] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2250.928311] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2253.293305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2259.968899] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2269.540416] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2278.903069] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2279.448389] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2287.330794] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2298.825508] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 2299.054391] Lustre: 59394:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 2299.086974] Lustre: 59394:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 2299.103989] Lustre: 59394:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2299.126410] Lustre: 59394:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2299.144876] Lustre: 59394:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2299.161089] Lustre: 59394:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2302.171448] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2309.938549] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 2318.283439] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 2329.584000] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2329.591189] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2329.626450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2334.689467] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2336.256516] Lustre: server umount lustre-OST0000 complete [ 2342.645632] Lustre: server umount lustre-MDT0000 complete [ 2360.031569] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 2366.334903] Key type lgssc unregistered [ 2366.769431] LNet: 61007:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2366.780298] LNetError: 61007:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2366.804225] LNet: Removed LNI 192.168.203.119@tcp [ 2368.444749] Key type .llcrypt unregistered [ 2368.449345] Key type ._llcrypt unregistered [ 2408.688696] Key type ._llcrypt registered [ 2408.690728] Key type .llcrypt registered [ 2408.870958] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2410.785614] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2410.847537] alg: No test for adler32 (adler32-zlib) [ 2412.086909] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 2412.519437] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 2414.247154] Key type lgssc registered [ 2416.382665] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2428.145840] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2428.180735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2429.712471] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2436.615582] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2447.192292] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2456.517180] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2457.145737] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2465.926842] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2474.482845] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 2474.598174] Lustre: 63848:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 514 < left 544, rollback = 9 [ 2474.608715] Lustre: 63848:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/1, destroy: 0/0/0 [ 2474.616849] Lustre: 63848:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 2474.625709] Lustre: 63848:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 2474.633633] Lustre: 63848:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 2474.643193] Lustre: 63848:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 2476.396340] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2484.512578] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 2500.068047] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2500.098311] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2501.727333] Lustre: server umount lustre-OST0000 complete [ 2508.820620] Lustre: server umount lustre-MDT0000 complete [ 2524.973299] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 2529.217867] Key type lgssc unregistered [ 2529.667293] LNet: 65424:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2529.680335] LNetError: 65424:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2529.723744] LNet: Removed LNI 192.168.203.119@tcp [ 2531.088835] Key type .llcrypt unregistered [ 2531.102815] Key type ._llcrypt unregistered [ 2558.635791] Key type ._llcrypt registered [ 2558.640111] Key type .llcrypt registered [ 2558.795473] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 2581.518372] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2583.026654] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2583.061511] alg: No test for adler32 (adler32-zlib) [ 2584.352481] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 2584.717674] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 2586.449064] Key type lgssc registered [ 2588.271163] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2640.396534] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2640.420669] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2641.727617] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2641.748961] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2641.810543] Lustre: lustre-MDT0000: new disk, initializing [ 2641.868439] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2641.895242] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2648.393537] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2661.617524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2661.884089] Lustre: 68585:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2662.010183] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2662.016161] Lustre: Skipped 1 previous similar message [ 2662.121295] Lustre: lustre-MDT0001: new disk, initializing [ 2662.322915] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2662.418453] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2662.447832] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2668.881840] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2687.633617] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2696.841988] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2706.909395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2707.356944] Lustre: lustre-OST0000: new disk, initializing [ 2707.369485] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2707.393927] Lustre: 70555:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2707.514377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2714.192343] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2714.206248] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2714.393454] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2716.669866] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2730.883238] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2738.629360] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2738.934108] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2745.731843] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2746.020481] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2750.434057] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2750.437429] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2750.472521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2750.490440] Lustre: Skipped 1 previous similar message [ 2755.291488] Lustre: server umount lustre-OST0000 complete [ 2760.306522] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2760.319749] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2760.324905] Lustre: Skipped 2 previous similar messages [ 2760.343879] Lustre: Skipped 1 previous similar message [ 2765.796403] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2765.811179] Lustre: Skipped 1 previous similar message [ 2766.968509] Lustre: server umount lustre-MDT0000 complete [ 2772.303467] LustreError: 69379:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785391050 with bad export cookie 1257466701979118482 [ 2772.306482] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2772.319586] LustreError: 69379:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2772.957592] Lustre: server umount lustre-MDT0001 complete [ 2785.335079] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 01:57:41 (1785391061) [ 2803.065682] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2863.123978] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 2877.194899] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 2925.235451] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2925.909609] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2926.049513] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2926.277511] Lustre: lustre-MDT0000: new disk, initializing [ 2926.482503] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2926.536095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2935.973898] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2950.666555] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2950.864292] Lustre: 76609:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2950.988195] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2950.994580] Lustre: Skipped 1 previous similar message [ 2951.120961] Lustre: lustre-MDT0001: new disk, initializing [ 2951.320632] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2951.404812] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2951.418443] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2958.549338] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 2970.833938] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2979.143674] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2989.274655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2989.777420] Lustre: lustre-OST0000: new disk, initializing [ 2989.789913] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2989.800157] Lustre: 78592:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2989.948386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2996.430768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2996.453314] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2996.649980] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2999.895170] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3012.731277] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3019.936915] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3020.302256] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3027.665397] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3028.004135] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3032.547370] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3032.554183] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3032.568238] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3032.573913] Lustre: Skipped 1 previous similar message [ 3032.604635] Lustre: Skipped 1 previous similar message [ 3037.664576] LustreError: 78839:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3037.681782] LustreError: 78839:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3038.000659] Lustre: server umount lustre-OST0000 complete [ 3047.923451] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3047.943548] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3047.955713] Lustre: Skipped 1 previous similar message [ 3048.013188] Lustre: Skipped 1 previous similar message [ 3050.263845] Lustre: server umount lustre-MDT0000 complete [ 3053.025134] LustreError: 76618:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3053.044498] LustreError: 76618:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3055.112297] LustreError: 76601:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785391332 with bad export cookie 1257466701979119357 [ 3055.121912] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3055.129936] LustreError: 76601:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3055.615266] Lustre: server umount lustre-MDT0001 complete [ 3069.361144] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:02:25 (1785391345) [ 3081.305659] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3082.379844] LustreError: 81005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3082.573046] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3087.863408] LustreError: 81006:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3091.465655] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3092.971561] LustreError: 81005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3103.205799] LustreError: 81006:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3103.257742] LustreError: 81006:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3105.982316] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3111.704248] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3122.816568] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3129.996162] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3140.041493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3140.650109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3140.654717] Lustre: Skipped 1 previous similar message [ 3148.574394] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3158.549181] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3162.658852] Lustre: 83681:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 68dc3871-c4a3-4f62-be41-6a8006605284 at adminstrative request [ 3163.623911] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3163.650895] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 3173.857594] Lustre: MGS: Client 68dc3871-c4a3-4f62-be41-6a8006605284 (at 0@lo) reconnecting [ 3173.874933] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x11736b4d9858f276 to 0x11736b4d9858f516 [ 3173.919365] Lustre: MGC192.168.203.119@tcp: Connection restored to 0@lo (at 0@lo) [ 3203.402851] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3203.723484] Lustre: lustre-OST0001: new disk, initializing [ 3203.732422] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3203.760512] Lustre: 84075:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3212.984290] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3213.980675] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3213.996982] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3214.190030] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3227.238036] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3234.788996] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3234.789919] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3234.806489] LustreError: Skipped 1 previous similar message [ 3234.807424] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 3234.835069] Lustre: Skipped 1 previous similar message [ 3237.053590] Lustre: server umount lustre-OST0001 complete [ 3239.907397] LustreError: 82831:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3239.930263] LustreError: 82831:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3244.000226] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3244.006866] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3248.202176] Lustre: server umount lustre-OST0000 complete [ 3254.753497] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3254.767122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3254.774459] Lustre: Skipped 4 previous similar messages [ 3261.032801] Lustre: server umount lustre-MDT0000 complete [ 3266.706442] LustreError: 80986:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785391544 with bad export cookie 1257466701979120918 [ 3266.716668] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3266.732840] LustreError: 80986:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3267.541324] Lustre: server umount lustre-MDT0001 complete [ 3278.735816] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:05:55 (1785391555) [ 3281.391646] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 3284.183250] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:06:00 (1785391560) [ 3286.225691] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 3289.043981] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:06:05 (1785391565) [ 3291.546981] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 3294.000554] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:06:10 (1785391570) [ 3296.042927] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 3298.492274] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:06:15 (1785391575) [ 3300.688871] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 3308.523964] loop: module loaded [ 3308.549654] loop0: detected capacity change from 0 to 4294967296 [ 3308.598686] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 3310.752066] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 3316.094986] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 3321.124929] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 3358.096400] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3358.152829] LustreError: 88207:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 3358.165875] LustreError: 88207:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 3358.175556] LustreError: 88207:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 3358.186431] LustreError: 88207:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 3382.811719] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 3399.813784] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 3450.376755] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3450.779746] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3450.843394] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3451.027980] Lustre: lustre-MDT0000: new disk, initializing [ 3451.154390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3451.162541] Lustre: Skipped 1 previous similar message [ 3451.177418] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3457.267241] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3469.650732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3469.817337] Lustre: 91096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3470.297739] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3477.435378] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3489.702868] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3497.322307] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3506.413381] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3506.838918] Lustre: 93070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3513.948363] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3514.150660] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3515.450329] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3527.967388] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3533.489912] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3533.759756] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3538.826209] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3539.134966] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3545.060899] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3545.065396] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3545.121088] Lustre: Skipped 3 previous similar messages [ 3545.159541] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3545.181823] Lustre: Skipped 4 previous similar messages [ 3548.348920] Lustre: server umount lustre-OST0000 complete [ 3560.995586] Lustre: server umount lustre-MDT0000 complete [ 3565.536673] LustreError: 92261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3565.584581] LustreError: 92261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 3569.099287] LustreError: 91091:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785391846 with bad export cookie 1257466701979121534 [ 3569.107771] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3569.136332] LustreError: 91091:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 3569.677945] Lustre: server umount lustre-MDT0001 complete [ 3587.877987] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:11:02 (1785391862) [ 3599.401888] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3600.855188] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 3600.861683] Lustre: Skipped 2 previous similar messages [ 3608.702954] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3626.059492] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3634.466338] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3647.131540] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3656.399744] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3667.474832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3676.499940] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3688.594877] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3696.489629] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3696.777256] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3704.604611] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3705.260292] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3717.382862] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3717.904532] Lustre: lustre-OST0001: new disk, initializing [ 3717.912373] Lustre: Skipped 2 previous similar messages [ 3717.929443] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3717.944701] Lustre: Skipped 3 previous similar messages [ 3717.971114] Lustre: 98859:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3724.019509] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 3724.031657] Lustre: Skipped 2 previous similar messages [ 3724.048201] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 3724.254376] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 3727.555775] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3739.446472] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3745.426478] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 3745.729427] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3752.645210] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 3753.041688] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3776.981218] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3810.336912] Lustre: Failing over lustre-MDT0000 [ 3810.810633] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3810.839717] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3810.855713] Lustre: Skipped 2 previous similar messages [ 3810.869499] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3810.881243] Lustre: Skipped 4 previous similar messages [ 3811.142884] Lustre: server umount lustre-MDT0000 complete [ 3811.314621] LustreError: 95483:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3811.335618] LustreError: 95483:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 3827.682309] Lustre: 66806:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785392089/real 1785392089] req@0000000069ec21d7 x1872118008830080/t0(0) o400->MGC192.168.203.119@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1785392105 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3827.725247] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3842.012432] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3842.019055] LDISKFS-fs (dm-0): recovery complete [ 3842.032700] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3852.258072] LustreError: 66802:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000e0b94903 x1872118008842112/t0(0) o250->MGC192.168.203.119@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3852.820604] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3855.935592] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3855.938290] Lustre: 95479:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3857.895746] Lustre: 95484:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3857.903743] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3857.916690] Lustre: 95484:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 3858.056972] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3858.392361] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3858.734330] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3858.782663] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3859.111069] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3859.123053] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 3859.144208] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3859.153661] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 3859.162814] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 7 previous similar messages [ 3859.800033] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3859.817390] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3860.161076] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3860.164417] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 3861.183888] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3861.195315] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 5 previous similar messages [ 3861.856913] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3861.863544] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3862.194035] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3862.211030] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 5 previous similar messages [ 3865.424202] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3865.440303] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 3866.081729] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3866.102551] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3866.423077] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3866.437453] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 3873.471953] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3873.483617] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 3874.139106] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3874.153948] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3874.495531] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3874.506517] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 3889.472983] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3889.496326] Lustre: 101715:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 49 previous similar messages [ 3890.145308] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3890.156373] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3890.799156] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3890.816649] LustreError: 101715:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 50 previous similar messages [ 3915.231559] Lustre: 101715:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3915.248776] Lustre: 101715:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3915.269205] Lustre: 101715:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client ffb74a06-df2d-40a8-8bb9-846ed93e0ae8@192.168.203.19@tcp [ 3915.287040] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3915.309862] Lustre: 101715:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@000000000c005624 x1872117986564480/t0(8589934775) o101->ffb74a06-df2d-40a8-8bb9-846ed93e0ae8@192.168.203.19@tcp:668/0 lens 592/0 e 0 to 0 dl 1785392203 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3915.361925] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3915.384286] Lustre: lustre-MDT0000: Denying connection for new client ffb74a06-df2d-40a8-8bb9-846ed93e0ae8 (at 192.168.203.19@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3915.407774] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3915.490711] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3915.501882] Lustre: Skipped 2 previous similar messages [ 3915.512324] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3915.544979] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:78 to 0x280000401:577) [ 3915.545706] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3938.788049] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3938.805497] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3938.832695] Lustre: Skipped 3 previous similar messages [ 3939.808779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3939.822686] Lustre: Skipped 1 previous similar message [ 3943.877480] Lustre: server umount lustre-OST0000 complete [ 3944.936670] LustreError: 97312:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3944.982802] LustreError: 97312:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 55 previous similar messages [ 3964.386110] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3965.078249] Lustre: server umount lustre-OST0001 complete [ 3977.735634] Lustre: server umount lustre-MDT0000 complete [ 3984.828932] LustreError: 95466:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785392262 with bad export cookie 1257466701979172746 [ 3984.833087] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3984.853568] LustreError: 95466:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3986.261538] Lustre: server umount lustre-MDT0001 complete [ 4001.488197] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 02:17:57 (1785392277) [ 4010.335207] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4010.347609] Lustre: *** cfs_fail_loc=197, val=0*** [ 4026.080145] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 4031.199537] LustreError: 103715:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.119@tcp: failed processing log, type 4: rc = -110 [ 4036.415872] Lustre: *** cfs_fail_loc=197, val=0*** [ 4036.417450] Lustre: Skipped 65 previous similar messages [ 4056.991266] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4057.005978] Lustre: Skipped 4 previous similar messages [ 4066.081719] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4075.961384] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4078.742979] Lustre: server umount lustre-OST0000 complete [ 4089.564457] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 02:19:26 (1785392366) [ 4114.487959] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4120.653078] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4133.277758] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4138.114504] LustreError: 105371:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 4138.133531] LustreError: 105371:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 4138.146574] LustreError: 105371:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 23, retries 0, failed: rc = -5 [ 4138.835339] Lustre: server umount lustre-MDT0000 complete [ 4181.229375] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 4194.154670] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 4241.486533] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4241.740357] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4241.760771] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4241.847228] Lustre: lustre-MDT0000: new disk, initializing [ 4241.943598] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4248.425308] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4265.730824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4265.946963] Lustre: 109546:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4266.702095] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4274.655255] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4290.868706] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4301.208428] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4312.975351] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4313.753780] Lustre: 111518:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4322.684571] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4324.890504] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4337.002108] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4345.266628] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4345.707930] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4352.018792] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4352.512037] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4358.626513] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4358.638744] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4358.649683] LustreError: Skipped 2 previous similar messages [ 4358.668615] Lustre: Skipped 5 previous similar messages [ 4358.680735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4358.680741] Lustre: Skipped 10 previous similar messages [ 4361.704147] Lustre: server umount lustre-OST0000 complete [ 4373.664744] Lustre: server umount lustre-MDT0000 complete [ 4374.010724] LustreError: 109558:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4374.031664] LustreError: 109558:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 13 previous similar messages [ 4379.584671] LustreError: 109539:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785392657 with bad export cookie 1257466701979183386 [ 4379.587233] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4379.600091] LustreError: 109539:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4380.423622] Lustre: server umount lustre-MDT0001 complete [ 4392.493441] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 02:24:28 (1785392668) [ 4411.628474] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 4418.284797] Key type lgssc unregistered [ 4419.133565] LNet: 114030:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4419.152047] LNetError: 114030:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4419.233616] LNet: Removed LNI 192.168.203.119@tcp [ 4421.570443] Key type .llcrypt unregistered [ 4421.599883] Key type ._llcrypt unregistered [ 4450.434693] Key type ._llcrypt registered [ 4450.439448] Key type .llcrypt registered [ 4450.679144] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 4481.971929] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 4484.717075] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4484.771695] alg: No test for adler32 (adler32-zlib) [ 4486.043877] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 4486.410424] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 4488.255805] Key type lgssc registered [ 4490.131854] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4553.599270] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4553.665193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4555.342732] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4555.414465] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4555.633229] Lustre: lustre-MDT0000: new disk, initializing [ 4555.786978] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4555.853057] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4562.413272] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4576.303628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4576.699489] Lustre: lustre-OST0000: new disk, initializing [ 4576.705415] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4576.715876] Lustre: Skipped 1 previous similar message [ 4576.723638] Lustre: 118213:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4576.895697] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4582.513593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 4582.529858] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 4582.589890] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 4586.110952] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4599.265526] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4599.540969] Lustre: lustre-OST0001: new disk, initializing [ 4599.555388] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 4599.572664] Lustre: 119078:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4599.696587] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 4608.242335] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4609.725127] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 4609.757243] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 4609.805498] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 4623.859641] Lustre: server umount lustre-MDT0000 complete [ 4632.778632] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 4640.488948] LustreError: 117401:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785392918 with bad export cookie 9632824047771424513 [ 4640.509910] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4641.759477] Lustre: 115418:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785392903/real 1785392903] req@00000000242e9f96 x1872120002931072/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785392919 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4641.802517] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4646.879126] Lustre: 115415:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785392908/real 1785392908] req@0000000081dfc5d9 x1872120002931456/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785392924 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4651.125288] Lustre: server umount lustre-OST0000 complete [ 4652.128220] Lustre: 115415:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785392913/real 1785392913] req@00000000f92534e6 x1872120002931840/t0(0) o400->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785392929 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 4657.452578] Lustre: server umount lustre-OST0001 complete [ 4676.505719] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 02:29:11 (1785392951) [ 4706.177823] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 02:29:42 (1785392982) [ 4718.311270] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 4751.301192] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 02:30:27 (1785393027) [ 4805.710107] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 4820.053450] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 4873.331120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4873.647110] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4873.678414] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4873.799505] Lustre: lustre-MDT0000: new disk, initializing [ 4873.917345] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4873.954080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4880.827498] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4894.375380] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4894.546516] Lustre: 126851:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4894.675183] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4894.684399] Lustre: Skipped 1 previous similar message [ 4894.840659] Lustre: lustre-MDT0001: new disk, initializing [ 4895.015165] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4895.071386] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4895.083706] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4900.308953] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4913.987760] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4922.345442] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4932.107826] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4932.502939] Lustre: lustre-OST0000: new disk, initializing [ 4932.508761] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4932.521960] Lustre: 128823:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4932.684080] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4940.402441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4940.405653] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4940.606879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4941.596423] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 4953.798688] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4960.636873] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4960.863297] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4965.505994] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4965.724812] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4971.488860] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4971.492392] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4971.514592] Lustre: Skipped 1 previous similar message [ 4971.536175] Lustre: Skipped 1 previous similar message [ 4974.760394] Lustre: server umount lustre-OST0000 complete [ 4981.729095] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4981.734438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4981.750029] Lustre: Skipped 1 previous similar message [ 4981.772982] Lustre: Skipped 1 previous similar message [ 4986.421332] Lustre: server umount lustre-MDT0000 complete [ 4991.364982] LustreError: 126842:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393269 with bad export cookie 9632824047771425738 [ 4991.366331] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4991.389564] LustreError: 126842:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4992.336644] Lustre: server umount lustre-MDT0001 complete [ 5004.076352] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 02:34:40 (1785393280) [ 5013.107856] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5013.921608] LustreError: 131237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5013.946084] LustreError: 131237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5014.060609] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5019.104562] LustreError: 131237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5019.125856] LustreError: 131237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5021.804521] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5023.206247] LustreError: 131238:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5028.324838] LustreError: 131237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5033.441261] LustreError: 131238:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5033.758020] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5042.068723] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5052.567578] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5059.449207] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5069.190054] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5069.742811] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5069.758465] Lustre: Skipped 1 previous similar message [ 5079.098936] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5089.204179] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5095.525349] Lustre: *** cfs_fail_loc=159, val=0*** [ 5111.961168] Lustre: lustre-MDT0000: Client 80535bb9-5e9e-4e32-b596-5871d96dfdd5 (at 192.168.203.19@tcp) reconnecting [ 5114.205780] Lustre: *** cfs_fail_loc=159, val=0*** [ 5114.212717] Lustre: Skipped 6 previous similar messages [ 5129.882262] Lustre: lustre-MDT0000: Client 80535bb9-5e9e-4e32-b596-5871d96dfdd5 (at 192.168.203.19@tcp) reconnecting [ 5140.449550] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5140.455397] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5140.466735] LustreError: Skipped 1 previous similar message [ 5140.470619] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5140.489875] Lustre: Skipped 1 previous similar message [ 5140.519500] Lustre: Skipped 1 previous similar message [ 5145.568814] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5150.692570] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5150.706019] Lustre: Skipped 2 previous similar messages [ 5151.200211] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5151.625880] Lustre: server umount lustre-OST0000 complete [ 5160.931341] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5160.936527] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5160.959559] Lustre: Skipped 1 previous similar message [ 5160.991718] Lustre: Skipped 1 previous similar message [ 5166.055258] LustreError: 133776:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5166.081333] LustreError: 133776:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5166.185680] Lustre: server umount lustre-MDT0000 complete [ 5171.495351] LustreError: 133843:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393449 with bad export cookie 9632824047771426620 [ 5171.499143] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5171.504337] LustreError: 133843:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5172.094636] Lustre: server umount lustre-MDT0001 complete [ 5193.081863] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 5201.000822] Key type lgssc unregistered [ 5201.628454] LNet: 135042:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5201.823854] LNetError: 135042:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5202.230609] LNet: Removed LNI 192.168.203.119@tcp [ 5205.537147] Key type .llcrypt unregistered [ 5205.538648] Key type ._llcrypt unregistered [ 5242.080288] Key type ._llcrypt registered [ 5242.084904] Key type .llcrypt registered [ 5242.176850] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 5243.408319] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5243.618815] alg: No test for adler32 (adler32-zlib) [ 5244.853313] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 5245.090034] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 5246.807134] Key type lgssc registered [ 5248.558029] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5259.163583] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5259.195432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5260.898027] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5268.244656] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5270.504218] LustreError: 136474:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5270.528451] LustreError: 136474:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5275.627689] LustreError: 136475:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5279.422823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5279.996883] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5286.246840] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5297.017778] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5305.013471] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5312.818294] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5313.331068] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5321.341925] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5331.078861] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5331.778137] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 5343.814768] Lustre: *** cfs_fail_loc=159, val=0*** [ 5360.244439] Lustre: lustre-MDT0000: Client 18095506-17fc-4e09-947f-cadcdb5e0a10 (at 192.168.203.19@tcp) reconnecting [ 5363.633931] Lustre: *** cfs_fail_loc=159, val=0*** [ 5363.641116] Lustre: Skipped 1 previous similar message [ 5380.394436] Lustre: lustre-MDT0001: Client 18095506-17fc-4e09-947f-cadcdb5e0a10 (at 192.168.203.19@tcp) reconnecting [ 5382.606079] Lustre: *** cfs_fail_loc=159, val=0*** [ 5382.607788] Lustre: Skipped 2 previous similar messages [ 5399.157244] Lustre: lustre-MDT0001: Client 18095506-17fc-4e09-947f-cadcdb5e0a10 (at 192.168.203.19@tcp) reconnecting [ 5407.711606] Lustre: 139518:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5411.878384] Lustre: *** cfs_fail_loc=159, val=0*** [ 5411.885189] Lustre: Skipped 4 previous similar messages [ 5428.374550] Lustre: lustre-MDT0000: Client 51e7664a-7aaf-4f01-b468-c22e60b5a4bd (at 192.168.203.19@tcp) reconnecting [ 5430.835360] Lustre: *** cfs_fail_loc=159, val=0*** [ 5430.836976] Lustre: Skipped 14 previous similar messages [ 5447.290492] Lustre: lustre-MDT0000: Client 51e7664a-7aaf-4f01-b468-c22e60b5a4bd (at 192.168.203.19@tcp) reconnecting [ 5449.760720] Lustre: 139809:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5459.938653] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5459.947244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5459.988971] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5460.001675] Lustre: Skipped 1 previous similar message [ 5465.060187] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5465.068684] Lustre: Skipped 1 previous similar message [ 5465.637584] Lustre: server umount lustre-OST0000 complete [ 5474.275644] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5474.292382] LustreError: Skipped 4 previous similar messages [ 5474.302628] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5474.330778] Lustre: Skipped 1 previous similar message [ 5474.347921] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5475.306390] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5477.846488] Lustre: server umount lustre-MDT0000 complete [ 5480.417060] LustreError: 139578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5480.448118] LustreError: 139578:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5483.356436] LustreError: 139417:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393761 with bad export cookie 8632122679525711189 [ 5483.374452] LustreError: 139417:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5483.377562] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5483.976881] Lustre: server umount lustre-MDT0001 complete [ 5500.756412] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 5504.552642] Key type lgssc unregistered [ 5505.082455] LNet: 140863:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5505.105966] LNetError: 140863:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5505.139829] LNet: Removed LNI 192.168.203.119@tcp [ 5506.443793] Key type .llcrypt unregistered [ 5506.448842] Key type ._llcrypt unregistered [ 5551.367631] Key type ._llcrypt registered [ 5551.369436] Key type .llcrypt registered [ 5551.541475] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 5553.923482] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5554.112807] alg: No test for adler32 (adler32-zlib) [ 5555.939863] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 5556.587449] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 5558.375193] Key type lgssc registered [ 5560.888721] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5576.352495] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5576.373584] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5577.652881] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5584.381366] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5587.940899] LustreError: 142281:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5587.954886] LustreError: 142281:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5593.064142] LustreError: 142282:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5597.397702] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5597.925917] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5604.163229] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5615.865202] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5627.076508] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5640.124735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5640.971358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5643.042083] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 5648.141849] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 5649.729508] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5660.037278] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5666.361861] Lustre: 144866:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5670.819098] Lustre: 144915:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5673.653348] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 5675.005264] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 5682.333245] Lustre: 145068:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 5683.992513] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 5685.483719] Lustre: 145173:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 5694.433723] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5694.435095] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5694.450579] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5694.457058] Lustre: Skipped 1 previous similar message [ 5699.059527] Lustre: server umount lustre-OST0000 complete [ 5704.672819] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5704.677851] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5704.692838] Lustre: Skipped 1 previous similar message [ 5704.705247] Lustre: Skipped 2 previous similar messages [ 5709.795295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5709.800996] Lustre: Skipped 1 previous similar message [ 5710.831996] Lustre: server umount lustre-MDT0000 complete [ 5714.920634] LustreError: 142274:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5714.947636] LustreError: 142274:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5715.967407] LustreError: 142262:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785393993 with bad export cookie 2190540377382942208 [ 5715.999611] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5716.000300] LustreError: 142262:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5716.779562] Lustre: server umount lustre-MDT0001 complete [ 5736.126969] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 5740.306118] Key type lgssc unregistered [ 5740.733631] LNet: 146250:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5740.745585] LNetError: 146250:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5740.779847] LNet: Removed LNI 192.168.203.119@tcp [ 5741.750415] Key type .llcrypt unregistered [ 5741.752052] Key type ._llcrypt unregistered [ 5778.019463] Key type ._llcrypt registered [ 5778.027853] Key type .llcrypt registered [ 5778.228762] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 5779.971515] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5780.089091] alg: No test for adler32 (adler32-zlib) [ 5781.666941] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 5782.040206] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 5783.783443] Key type lgssc registered [ 5786.006376] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5799.087875] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5799.175227] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5801.334106] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5809.883053] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5811.692171] LustreError: 147675:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5811.727064] LustreError: 147675:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5816.807361] LustreError: 147674:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5821.507675] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5821.944293] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5827.344957] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5841.036030] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5852.495424] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5862.717959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5863.271459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5867.434947] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 5871.518333] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 5872.482621] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 5882.715871] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5890.575662] Lustre: *** cfs_fail_loc=159, val=0*** [ 5907.093888] Lustre: lustre-MDT0000: Client 94c57785-03c3-4ad2-8a87-014dd67474e8 (at 192.168.203.19@tcp) reconnecting [ 5913.569239] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5913.583374] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5913.596640] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5917.665502] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5917.668933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5917.688663] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5917.704405] Lustre: Skipped 1 previous similar message [ 5919.826257] Lustre: server umount lustre-OST0000 complete [ 5927.907216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5927.927726] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5927.935658] Lustre: Skipped 1 previous similar message [ 5933.029994] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5933.035532] Lustre: Skipped 1 previous similar message [ 5938.146723] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5938.153919] Lustre: Skipped 1 previous similar message [ 5939.167135] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5939.989728] Lustre: server umount lustre-MDT0000 complete [ 5942.262115] LustreError: 147669:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5942.284297] LustreError: 147669:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5945.247989] LustreError: 147656:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394223 with bad export cookie 11165077330380617708 [ 5945.252709] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5945.261410] LustreError: 147656:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5945.581657] Lustre: server umount lustre-MDT0001 complete [ 5963.096662] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 5968.733971] Key type lgssc unregistered [ 5969.203660] LNet: 151362:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5969.212581] LNetError: 151362:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5969.233213] LNet: Removed LNI 192.168.203.119@tcp [ 5970.432662] Key type .llcrypt unregistered [ 5970.435626] Key type ._llcrypt unregistered [ 6003.706212] Key type ._llcrypt registered [ 6003.708218] Key type .llcrypt registered [ 6003.829642] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 6005.522290] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6005.587099] alg: No test for adler32 (adler32-zlib) [ 6007.069534] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 6007.496384] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 6009.295826] Key type lgssc registered [ 6010.476246] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6020.881475] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6020.931488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6022.811914] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6029.655165] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6032.355663] LustreError: 152784:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6032.385880] LustreError: 152784:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6037.480714] LustreError: 152785:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6042.612097] LustreError: 152784:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6043.848976] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6044.324949] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6052.916386] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6069.897046] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6078.311402] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6089.200671] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6089.395421] LustreError: 154616:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6090.035542] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6093.154974] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 6093.172126] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 6101.787670] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6115.635416] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6139.120631] Lustre: lustre-MDT0001: evicting 7014d779-b584-4607-999d-1813d2b9c3b5 (at 192.168.203.19@tcp) by administrative request [ 6139.130738] Lustre: Skipped 1 previous similar message [ 6175.200973] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6175.224510] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6175.235578] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6175.250272] Lustre: Skipped 1 previous similar message [ 6175.269546] Lustre: Skipped 1 previous similar message [ 6176.771242] Lustre: server umount lustre-OST0000 complete [ 6182.374882] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6182.386037] LustreError: Skipped 1 previous similar message [ 6182.394842] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6182.431901] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6185.440889] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6185.442762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6185.477871] Lustre: Skipped 1 previous similar message [ 6188.700655] Lustre: server umount lustre-MDT0000 complete [ 6190.563076] LustreError: 152784:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6190.589110] LustreError: 152784:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6194.653709] LustreError: 152763:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394472 with bad export cookie 12015016063628867188 [ 6194.669337] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6195.651415] Lustre: server umount lustre-MDT0001 complete [ 6220.051280] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 6227.226531] Key type lgssc unregistered [ 6227.926125] LNet: 156886:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6227.970530] LNetError: 156886:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6228.037279] LNet: Removed LNI 192.168.203.119@tcp [ 6230.669483] Key type .llcrypt unregistered [ 6230.671141] Key type ._llcrypt unregistered [ 6361.685904] Key type ._llcrypt registered [ 6361.693625] Key type .llcrypt registered [ 6362.070990] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 6363.923277] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6364.034545] alg: No test for adler32 (adler32-zlib) [ 6365.424688] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 6365.963173] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 6367.807163] Key type lgssc registered [ 6370.147271] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6380.932368] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6380.953264] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6382.816419] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6388.410321] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6392.290545] LustreError: 159387:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6392.320454] LustreError: 159387:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6397.417520] LustreError: 159386:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6402.222100] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6402.531342] LustreError: 159387:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6403.151787] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6409.621684] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6421.811736] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6431.452531] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6443.132029] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6443.924351] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6447.997123] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 6453.139670] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 6453.413805] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6467.657200] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6487.244413] Lustre: server umount lustre-OST0000 complete [ 6494.181724] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6494.203627] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6494.208318] Lustre: Skipped 1 previous similar message [ 6499.297746] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6499.315419] Lustre: Skipped 1 previous similar message [ 6503.738103] Lustre: server umount lustre-MDT0000 complete [ 6504.415429] Lustre: 158698:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785394766/real 1785394766] req@00000000d22d99fa x1872121973215232/t0(0) o400->lustre-OST0000-osc-MDT0001@0@lo:28/4 lens 224/224 e 0 to 1 dl 1785394782 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 6504.427782] LustreError: 160286:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6504.432104] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6504.432114] Lustre: Skipped 1 previous similar message [ 6504.511466] Lustre: 158698:0:(client.c:2490:ptlrpc_expire_one_request()) Skipped 1 previous similar message [ 6504.557087] LustreError: 160286:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6508.928934] LustreError: 159368:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785394786 with bad export cookie 16359515651158102313 [ 6508.933531] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6508.959585] LustreError: 159368:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6510.105685] Lustre: server umount lustre-MDT0001 complete [ 6526.498586] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 6530.843994] Key type lgssc unregistered [ 6531.147240] LNet: 162956:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6531.170150] LNetError: 162956:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6532.224560] LNet: Removed LNI 192.168.203.119@tcp [ 6534.122552] Key type .llcrypt unregistered [ 6534.124940] Key type ._llcrypt unregistered [ 6567.182276] Key type ._llcrypt registered [ 6567.185848] Key type .llcrypt registered [ 6567.360623] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 6569.062867] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6569.139304] alg: No test for adler32 (adler32-zlib) [ 6570.416084] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 6570.802765] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 6572.559215] Key type lgssc registered [ 6574.607676] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6595.232397] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 03:01:11 (1785394871) [ 6620.285157] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [ 6637.133229] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 6689.498573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6689.530713] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6691.187984] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6691.276578] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6691.490654] Lustre: lustre-MDT0000: new disk, initializing [ 6691.746759] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6691.807440] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6698.692648] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6713.218088] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6713.393562] Lustre: 167258:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6713.485126] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6713.489678] Lustre: Skipped 1 previous similar message [ 6713.689543] Lustre: lustre-MDT0001: new disk, initializing [ 6713.882832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6713.957783] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6713.989225] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6720.874921] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6735.194890] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6742.099275] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6752.390028] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6752.817248] Lustre: lustre-OST0000: new disk, initializing [ 6752.824828] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6752.833621] Lustre: 169232:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6753.013841] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6761.700075] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6762.595656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6762.616064] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6762.825591] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6776.703867] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6788.583450] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6788.597797] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6788.604340] LustreError: Skipped 1 previous similar message [ 6788.607195] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6788.627768] Lustre: Skipped 1 previous similar message [ 6793.643677] Lustre: server umount lustre-OST0000 complete [ 6800.870463] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6800.884507] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6800.910583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6800.924063] Lustre: Skipped 1 previous similar message [ 6803.944609] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6803.956764] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6803.977257] Lustre: Skipped 1 previous similar message [ 6805.885768] Lustre: server umount lustre-MDT0000 complete [ 6810.847430] LustreError: 167251:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785395088 with bad export cookie 12742042960536809508 [ 6810.856525] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6810.872352] LustreError: 167251:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6811.652325] Lustre: server umount lustre-MDT0001 complete [ 6829.182217] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 6834.661029] Key type lgssc unregistered [ 6835.080818] LNet: 171160:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6835.091687] LNetError: 171160:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6835.131693] LNet: Removed LNI 192.168.203.119@tcp [ 6836.715738] Key type .llcrypt unregistered [ 6836.722737] Key type ._llcrypt unregistered [ 6871.326319] Key type ._llcrypt registered [ 6871.331237] Key type .llcrypt registered [ 6871.542851] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [ 6873.284496] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6873.394623] alg: No test for adler32 (adler32-zlib) [ 6874.655442] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [ 6875.040614] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [ 6876.911159] Key type lgssc registered [ 6878.515242] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6889.450481] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6889.462778] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6891.205479] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6898.451374] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6900.705524] LustreError: 172579:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6900.719393] LustreError: 172579:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6905.831716] LustreError: 172580:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6908.979074] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6909.767236] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6916.856117] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6926.224869] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6934.802916] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6945.199730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6945.754072] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6955.005417] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [ 6961.164863] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 6965.238410] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6973.677556] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6973.753337] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6973.799899] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6973.830841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6973.858887] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6975.533973] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6975.571386] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6975.607861] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6975.633594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6975.658564] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6977.352114] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6977.406229] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6977.445353] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6977.455173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6977.473307] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6978.854714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.000910] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6980.012270] Lustre: Skipped 1 previous similar message [ 6980.040453] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6980.056512] Lustre: Skipped 1 previous similar message [ 6980.064532] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6980.072230] Lustre: Skipped 1 previous similar message [ 6980.087704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6980.134263] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6980.145756] Lustre: Skipped 1 previous similar message [ 6981.311850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6982.701137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6983.885937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.373924] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6985.387601] Lustre: Skipped 3 previous similar messages [ 6985.413614] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6985.436398] Lustre: Skipped 3 previous similar messages [ 6985.449958] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6985.458066] Lustre: Skipped 3 previous similar messages [ 6985.465775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6985.480191] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6985.495592] Lustre: Skipped 3 previous similar messages [ 6986.552656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6987.889838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6989.247823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6990.563473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6991.910807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6993.116256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6994.400765] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6994.409941] Lustre: Skipped 6 previous similar messages [ 6994.431262] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6994.441865] Lustre: Skipped 6 previous similar messages [ 6994.446940] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6994.451772] Lustre: Skipped 6 previous similar messages [ 6994.461834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6994.476625] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6994.483641] Lustre: Skipped 6 previous similar messages [ 6995.399643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6996.336897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6997.495827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6998.749352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.040822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7000.867350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7002.027322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7003.395859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7004.726040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7006.098535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7007.501771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7008.785222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7010.118139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.324783] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7011.339060] Lustre: Skipped 13 previous similar messages [ 7011.374498] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7011.394083] Lustre: Skipped 13 previous similar messages [ 7011.399570] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7011.406214] Lustre: Skipped 13 previous similar messages [ 7011.413182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7011.418788] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7011.426578] Lustre: Skipped 13 previous similar messages [ 7012.930058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7014.164331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7015.220139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7016.317547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7017.520539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7018.905159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7019.982642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.241303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7022.629470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7023.946695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7025.259174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7026.539910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7027.828649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7029.015904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7030.255493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7031.374168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7032.381800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7033.222402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7034.305045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7035.373085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7036.472279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7037.556548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7038.490324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7039.569168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7040.694132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7041.844231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7042.851291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7044.121440] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7044.138271] Lustre: Skipped 27 previous similar messages [ 7044.168488] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7044.184122] Lustre: Skipped 27 previous similar messages [ 7044.192910] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7044.195902] Lustre: Skipped 27 previous similar messages [ 7044.211079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7044.227657] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7044.230254] Lustre: Skipped 27 previous similar messages [ 7045.236534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7046.550897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7047.783904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7048.976069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7050.171810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7051.520360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7052.650635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7053.877747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7054.969116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7056.217966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7057.286917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7058.288590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7059.432083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7060.600053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7061.585161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7062.781355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7063.861800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7064.940550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7066.105223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7067.327242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7068.586248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7069.846755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7071.213543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7072.648073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7073.968467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7074.948402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7076.399081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7077.569615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7078.869941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7080.033800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7081.597680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7083.475760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7084.706426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7086.047328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7087.398713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7088.429375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7090.226380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7091.810926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7093.688909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7095.377836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7097.125673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7098.800708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7099.969165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7101.018719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7102.432789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7103.766790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7105.204077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7106.674355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7107.997515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.234154] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7109.243843] Lustre: Skipped 49 previous similar messages [ 7109.261350] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7109.282110] Lustre: Skipped 49 previous similar messages [ 7109.288618] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7109.299466] Lustre: Skipped 49 previous similar messages [ 7109.313947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7109.335242] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7109.340899] Lustre: Skipped 49 previous similar messages [ 7111.019304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7112.486442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7113.856101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7115.070697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7116.985929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7118.783965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7119.938385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7121.173969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7123.051768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.966526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7127.177189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7129.081559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7130.678214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7133.223846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.100363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7136.388402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7137.689878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7139.743067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7140.858188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7142.133956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7143.533918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7144.698758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7146.192712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7147.611728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7149.531569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7151.088307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7153.188473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7154.691735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7155.990072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7157.677155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7159.448660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7160.543399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7162.027247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7163.405675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7164.608783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7165.895283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7167.025060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7168.451858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7169.814124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7171.236230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7172.519380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7173.776687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7174.944302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7176.290383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.527163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7178.918946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7180.180739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7181.406874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7182.728464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7183.886760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7185.277302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7186.526959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7187.952578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7189.242254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7190.497526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7191.766588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7193.110804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7194.277913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7195.237457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7196.352592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7197.462279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7198.646238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7200.051172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7201.303367] LustreError: 172599:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 7201.327387] LustreError: 172599:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 7201.351211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7202.602799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7203.771589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7205.025848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7206.342972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7207.538162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7208.943599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7210.269085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7211.710538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7213.076283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7214.456336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7215.566089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7216.753686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7217.816890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7219.101612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7220.381316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7221.561131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7222.475371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7223.619659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7224.812185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7225.920888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7227.244031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7228.451301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7229.672872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7231.013489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7232.150447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7233.410689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7234.675528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7235.785123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7237.042874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.335742] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7238.351187] Lustre: Skipped 93 previous similar messages [ 7238.391474] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7238.418584] Lustre: Skipped 93 previous similar messages [ 7238.427248] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7238.437195] Lustre: Skipped 93 previous similar messages [ 7238.444132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7238.464956] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7238.467619] Lustre: Skipped 93 previous similar messages [ 7239.778438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7241.312668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7242.653692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7243.750287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7244.965626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7246.223374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7247.299039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7248.572580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7249.796980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7251.108069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7252.355282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7253.582551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7255.031903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7256.343903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7257.438636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7258.713170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7259.750516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7260.848764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.154367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7262.924899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7263.971287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7265.273892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7266.543408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7268.067194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7269.367586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7270.706218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7271.970603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7273.350518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7274.894097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7277.008369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7278.357139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7279.984811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7281.216119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7282.557916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7284.098672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7285.755880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7287.384302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7288.978289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7290.669554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7292.711410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7294.570182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7295.859229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7297.524302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7299.170375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7300.789278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7302.252079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7303.939130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7305.569824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7307.266072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7309.129962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7310.525927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7312.093441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7313.603963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7314.990587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7316.451111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7318.009531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7319.157918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7320.823501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7322.306742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7323.641131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7324.750078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7326.415262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7327.931412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7329.844289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7331.810994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7333.054692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7334.031639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7335.419966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7336.461702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7337.341532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7338.202816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.112510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7339.936117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7341.130810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7342.236611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7343.446979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7344.779457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7345.895651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7347.022276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7348.519378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7349.652513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7350.914969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7352.411931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7354.058511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7355.690647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7357.115545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7358.384723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7359.674921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7360.890293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7362.081474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7363.264300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7364.959254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7366.366625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7367.877221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7369.380402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7370.564085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7371.751329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7373.374816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7375.088811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7376.465544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7377.858647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7379.052254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7380.348596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7381.711891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7383.416372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7385.263491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7386.729215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7387.835256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7389.037322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7390.554107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7392.052487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7393.637629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7395.262794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7396.695701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7398.044450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7399.481121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7400.749127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7401.918165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7403.311426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7404.469813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7405.546669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7406.645048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7407.683425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7408.582514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7409.879266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7410.962800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7412.221241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7413.470861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7414.768159] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7416.025704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7417.261131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7418.358211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7419.548143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7420.774125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7421.886015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7422.893796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.043377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7424.968428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7426.295064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7427.569948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7428.717027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7430.056991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7431.272845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7432.383217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7433.620329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7434.940629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7436.148188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7437.482320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7438.534655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7439.782494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7441.215283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7442.415276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7443.536041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7444.591816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7445.664190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7446.678497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7447.868447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7448.747176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7450.074523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7451.201379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7452.254781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7453.224862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7454.454706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7455.508697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7456.660051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7457.828905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7459.008327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7460.008076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7461.250634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7462.155339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7463.254482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7464.342412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7465.419288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7466.870380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7467.916022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7468.897025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7469.954030] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7470.970983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7472.138541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7473.425209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7474.778783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.031563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7476.986521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7478.206418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7479.257254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7480.537334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7481.759953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7483.051105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7484.627214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7486.201298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7487.740228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7489.385993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7491.188162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7492.314743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7493.842160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.275684] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7495.292080] Lustre: Skipped 195 previous similar messages [ 7495.319608] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7495.329522] Lustre: Skipped 195 previous similar messages [ 7495.336763] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7495.349406] Lustre: Skipped 195 previous similar messages [ 7495.369667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7495.388161] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7495.400552] Lustre: Skipped 195 previous similar messages [ 7496.983513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7498.319563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7499.694884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7501.012565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7502.282201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7503.328075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7504.586320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7506.070354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7507.332535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7508.693525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7509.774315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7510.777344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7512.012715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7513.200985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7514.511950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7515.662463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7516.838477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7518.086497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7519.352183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7520.726913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7521.988628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7523.012149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7524.310093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7525.795985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7526.789273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7528.041147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7529.553643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7530.746471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7532.733501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7534.300477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7535.824478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7537.170461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7538.815302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7540.764767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7542.470175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7544.238219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7546.095234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7547.727722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7549.719653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7551.721589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7553.632094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7555.315762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7556.764789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7558.459990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7560.044197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7561.570875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7562.774287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7563.897526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7565.024328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7566.115380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7567.476537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7568.887959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7570.166438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7571.382733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7572.397722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7573.895466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7575.968834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7577.439805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7578.905800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7580.464772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7582.004519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7583.347371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7584.318754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7585.662658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7586.833576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7587.895454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7588.869825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7589.877241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7590.717154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7592.135740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7593.238097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7594.422837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7595.785649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7596.784980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7597.795758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7598.897621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7599.942073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7600.964373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7602.285269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7603.640897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7604.913225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7606.155496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7607.032330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7608.010308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7609.164395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7610.568386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7611.622447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7612.807041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7613.978646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7615.386886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7616.471601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7617.503564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7618.817805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7619.889356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7621.150807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7622.642686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7624.055427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7625.193375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7626.231381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7627.560937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7628.897980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7629.914706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7630.775239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7631.626677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7632.820233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7633.973856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7635.137181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7636.235444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7637.132541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7638.282691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7639.854554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7640.799561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7641.883974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7643.144643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7644.117672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7645.444541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7646.560174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7647.674456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7648.879204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7650.003993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7651.097352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7652.321022] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7653.550317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7654.633193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7655.865950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7657.184269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7658.432919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7659.706552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7660.978265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7662.472572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7663.764177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7664.865714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7666.043052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7667.079604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7668.242336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7669.409891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7670.653220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7672.105934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7673.165046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7674.328957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7675.618394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7676.923235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7678.365268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7679.584275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7680.888784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7681.826295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7682.807614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7683.825578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7685.187138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7686.714671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7688.512552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7690.414957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7692.283736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7693.989586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7695.860517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7697.512132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7699.022778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7700.352070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7701.465112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7703.331174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7704.587777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7705.714220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7706.740200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7708.087949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7709.512726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7710.811065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7712.145747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7713.654167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7714.945822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7716.099626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7717.586827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7718.784795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7719.871907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7721.388134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7722.877236] LustreError: 172599:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -108 [ 7722.888659] LustreError: 172599:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -108 [ 7722.922734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7724.066711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7725.417911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7726.947300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7728.171724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7729.275521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7730.732350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7732.076834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7733.834716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7735.316216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7736.802247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7738.172948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7739.765034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7741.200868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7742.541434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7744.229757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7745.771971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7746.804983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7748.126645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7749.700065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7750.913977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7752.008200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7753.354627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7754.764573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7756.046471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7757.466950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7758.922449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7760.254417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7762.037184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7763.443294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7764.560874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7765.753132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7766.838730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7767.844848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7769.136201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7770.371111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7771.543549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7772.949221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7774.116088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7775.216464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7776.474519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7777.592827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7778.772179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7780.182993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7781.345645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7782.521963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7784.200953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7785.631548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7786.997367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7788.512279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7789.681416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7790.870787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7791.930281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7792.938766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7793.929439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7795.178576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7796.239255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7797.182852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7798.290419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7799.218447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7800.584980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7801.790960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7803.290421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7804.526994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7805.632807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7806.960150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7808.271185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7809.218643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7811.101979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7812.502733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7813.953986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7815.557283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7816.376054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7817.497933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7818.808507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7819.883484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7820.665535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7821.653092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7822.496495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7824.117671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7825.573123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7826.875994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7828.176180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7829.324283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7830.700590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7832.015125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7833.311563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7834.640571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7835.957043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7837.214041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7838.757335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7840.305219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7841.530956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7843.117761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7844.244628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7845.488471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7846.671502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7847.966398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7849.318198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7850.611101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7851.930148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7853.161358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7854.408353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7855.750033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7856.958988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7858.202681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7859.732394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7860.855692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7862.080893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7863.543949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7865.053614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7866.282934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7867.826371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7869.530332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7870.787210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7872.065760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7873.601145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7875.076573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7876.297083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7877.562711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7878.885565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7880.166771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7881.450970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7882.499682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7883.988543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7885.593111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7887.038364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7888.036421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7889.427215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7891.021081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7892.451678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7893.768315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7895.244874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7896.670239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7897.829553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7899.176991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7900.578827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7901.750380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7902.850188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7904.156901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7905.607485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7906.757797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7907.866920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7909.353717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7910.621738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7911.554780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7912.709849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7913.992052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7915.154916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7916.198650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7917.289943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7918.695870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7919.881689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7921.148500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7922.491908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7923.930165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7925.159639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7926.376164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7927.804340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7929.308940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7930.518082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7931.700367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7932.970713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7934.139890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7935.439829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7936.738280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7937.928148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7939.392192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7940.829303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7942.099241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7943.165932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7944.581323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7946.130486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7947.305472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7948.544309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7949.539222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7950.970687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7952.042513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7953.421703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7954.962074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7956.321035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7957.965926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7959.230423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7960.766605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7962.328972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7963.984544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7965.480031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7966.623640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7967.638778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7968.410669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7969.408799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7970.502858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7971.830280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7973.446104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7974.986119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7976.550537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7978.000650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7979.336487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7980.931626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7982.423355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7984.347395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7986.391944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7987.724930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7989.249583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7990.600166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7991.553801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7992.874034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7994.119842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7995.329410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7996.598553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7997.929594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7999.666100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8001.224622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8002.457663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8003.964935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8005.347629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8006.527732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8007.968589] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8007.980632] Lustre: Skipped 391 previous similar messages [ 8008.010746] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8008.025762] Lustre: Skipped 391 previous similar messages [ 8008.034967] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8008.043794] Lustre: Skipped 391 previous similar messages [ 8008.061416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8008.083122] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8008.086190] Lustre: Skipped 391 previous similar messages [ 8009.481632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8010.814337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8012.849384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8014.541719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8015.932106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8017.579493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8019.066451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8021.228064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8023.876721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8025.441061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8027.328348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8028.972324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8030.537124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8031.954060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8033.660148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8035.401249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8036.998702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8038.258284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8039.858340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8041.623101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8043.289946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8044.810671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8045.925168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8047.371803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8048.775151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8050.123798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8051.593792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8053.756370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8056.015837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8057.688844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8059.477139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8061.401554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8063.307578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8064.859486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8066.151367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8067.345604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8068.575943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8069.765186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8070.941986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8072.181772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8073.490667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8074.785967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8076.287784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8077.624248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8078.843992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8079.951418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8081.424400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8082.676837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8083.993384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8085.297193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8086.524217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8087.651495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8088.572935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8089.660689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8090.768150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8091.934854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8093.169693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8094.445538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8095.599661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8096.869840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8098.029066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8099.142563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8100.291189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8101.486342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8102.611451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8103.706831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8104.769071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8105.872535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8107.371669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8108.874995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8110.253836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8111.581874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8112.794828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8113.840485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8115.063489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8116.328627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8117.419519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8118.609939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8119.816830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8120.823234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8121.904811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8123.223745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8124.315502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8125.586117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8126.910162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8128.335058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8129.519410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8130.702916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8131.781045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8133.121925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8134.325171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8135.537378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8136.682478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8137.794702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8139.016724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8140.215935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8141.436969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8142.670803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8143.718209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8144.847666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8146.304801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8147.583971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8148.795297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8149.987830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8150.973792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8151.989933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8152.946250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8154.154776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8155.260428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8156.533846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8157.679151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8159.008983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8160.237193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8161.438354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8162.593409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8163.764936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8164.933239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8166.367284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8167.660407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8168.934219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8170.242074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8171.603792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8172.865116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8174.160587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8175.819271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8177.398768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8178.813748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8180.310938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8181.807916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8183.284972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8185.318454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8187.189235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8188.536985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8189.776741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8190.894941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8192.278570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8193.621083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8195.014600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8196.398933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8198.139102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8199.854749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8201.375487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8203.120964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8204.287139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8205.503593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8207.299706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8209.285575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8211.069976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8213.308556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8215.658079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8218.324146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8220.726696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8222.421240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8224.084238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8225.735401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8227.157414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8228.769647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8230.802484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8232.121882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8233.751543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8235.187632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8236.350152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8237.375717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8238.695652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8240.063838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8241.475820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8243.074100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8244.744029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8246.358812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8247.800974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8249.070303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8250.305070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8251.490972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8252.684316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8254.066531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8255.410679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8256.632138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8257.919959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8259.034794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8260.231442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8261.840605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8263.301863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8264.837789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8266.127559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8267.372681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8268.866558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8270.345612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8271.616993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8273.020650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8274.274744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8275.352509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8276.248028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8277.452100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8278.718487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8279.990611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8281.210212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8282.232975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8283.387287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8284.724401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8286.055131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8287.595367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8288.992850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8290.130977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8291.475584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8292.675760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8293.709122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8295.038640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8296.270665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8297.639311] LustreError: 172599:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 8297.653551] LustreError: 172599:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 8297.666915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8299.040279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8972.134522] LustreError: 174403:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 8972.154972] LustreError: 172599:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 9951.200860] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9951.218212] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9951.260924] Lustre: Skipped 210 previous similar messages [ 9951.283271] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9952.226639] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9952.231668] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9954.283797] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9954.302827] Lustre: Skipped 1 previous similar message [ 9957.155984] Lustre: server umount lustre-OST0000 complete [ 9964.513856] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9964.522049] Lustre: Skipped 1 previous similar message [ 9969.660993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9969.673539] Lustre: Skipped 1 previous similar message [ 9970.584165] Lustre: server umount lustre-MDT0000 complete [ 9974.753261] LustreError: 172575:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9974.796369] LustreError: 172575:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9976.086263] LustreError: 172560:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398253 with bad export cookie 11833600714118375708 [ 9976.088597] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9976.101417] LustreError: 172560:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9977.087165] Lustre: server umount lustre-MDT0001 complete [ 9994.727670] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [ 9999.189889] Key type lgssc unregistered [ 9999.818170] LNet: 227992:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9999.844672] LNetError: 227992:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9999.881767] LNet: Removed LNI 192.168.203.119@tcp [10001.314585] Key type .llcrypt unregistered [10001.324487] Key type ._llcrypt unregistered [10037.816641] Key type ._llcrypt registered [10037.823033] Key type .llcrypt registered [10038.032849] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [10039.628749] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10039.679668] alg: No test for adler32 (adler32-zlib) [10041.115575] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [10041.379063] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [10043.007624] Key type lgssc registered [10045.083740] Lustre: Echo OBD driver; http://www.lustre.org/ [10057.919056] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10057.960737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10059.718902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10066.824993] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10069.986971] LustreError: 229408:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10070.002874] LustreError: 229408:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10075.110083] LustreError: 229409:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10078.409553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10079.106475] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10085.768845] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10095.951442] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10105.036833] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10115.366948] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10115.997061] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10124.988807] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10129.424838] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [10137.084872] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10144.596947] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [10146.974012] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10147.002806] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [10147.014251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10147.035872] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [10147.041304] LustreError: 229428:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [10149.878686] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10149.884766] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10149.906474] Lustre: Skipped 1 previous similar message [10149.934695] Lustre: Skipped 1 previous similar message [10150.501588] Lustre: lustre-OST0000: Not available for connect from 192.168.203.19@tcp (stopping) [10154.976208] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10154.986419] Lustre: Skipped 1 previous similar message [10155.576324] LustreError: 231723:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 192.168.203.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [10155.852697] Lustre: server umount lustre-OST0000 complete [10167.055184] LustreError: 229428:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [10194.299234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10194.900040] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10204.720053] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10205.170557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10205.201865] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [10205.204551] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [10205.227201] Lustre: Skipped 1 previous similar message [10218.834455] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10220.957314] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [10229.195648] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92a389fa2000.ost_server_uuid 50 [10231.351187] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92a389fa2000.ost_server_uuid in IDLE state after 0 sec [10240.996185] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10241.036134] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10241.039962] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10241.056571] Lustre: Skipped 1 previous similar message [10244.632696] Lustre: server umount lustre-OST0000 complete [10251.256550] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10251.286414] Lustre: Skipped 1 previous similar message [10251.299438] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10251.310130] Lustre: Skipped 1 previous similar message [10256.885455] Lustre: server umount lustre-MDT0000 complete [10260.464367] LustreError: 229403:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10260.483548] LustreError: 229403:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10262.645498] LustreError: 229391:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398540 with bad export cookie 7152424222498089535 [10262.659948] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10262.666701] LustreError: 229391:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [10263.971603] Lustre: server umount lustre-MDT0001 complete [10282.948548] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [10289.955973] Key type lgssc unregistered [10290.758792] LNet: 234637:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10290.763315] LNetError: 234637:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10290.777638] LNet: Removed LNI 192.168.203.119@tcp [10292.371098] Key type .llcrypt unregistered [10292.376238] Key type ._llcrypt unregistered [10342.645025] Key type ._llcrypt registered [10342.648093] Key type .llcrypt registered [10342.838864] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [10344.291394] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10344.642692] alg: No test for adler32 (adler32-zlib) [10346.050686] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [10346.332792] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [10348.104071] Key type lgssc registered [10349.967950] Lustre: Echo OBD driver; http://www.lustre.org/ [10362.943256] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10362.967288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10364.647158] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10371.019275] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10375.154959] LustreError: 236371:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10375.182774] LustreError: 236371:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10380.264251] LustreError: 236372:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10383.516181] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10384.234354] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10391.409336] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10395.768941] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [10404.940380] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10405.402219] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10409.542526] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [10413.623487] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10424.997649] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10425.517597] Lustre: lustre-OST0001: new disk, initializing [10425.527291] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10425.536084] Lustre: 238681:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10425.725382] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10435.299960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10435.338722] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10435.579208] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10436.430513] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10452.034954] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10458.732139] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [10486.752213] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10486.757162] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10486.759547] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10486.798563] Lustre: Skipped 3 previous similar messages [10491.613958] Lustre: server umount lustre-MDT0000 complete [10491.879304] LustreError: 236372:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10496.622836] LustreError: 239451:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398774 with bad export cookie 10056934334345836682 [10496.627211] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10496.656300] LustreError: 239451:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [10497.004917] LustreError: 239261:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10497.035348] LustreError: 239261:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10497.043642] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10497.389392] Lustre: server umount lustre-MDT0001 complete [10503.261387] Lustre: server umount lustre-OST0000 complete [10509.935267] Lustre: server umount lustre-OST0001 complete [10550.735698] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [10568.353100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10569.608652] LustreError: 241797:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10569.657617] LustreError: 241797:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10569.786869] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10576.718514] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10579.938982] LustreError: 241798:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10579.985583] LustreError: 241798:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10591.087283] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10591.876199] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10600.790790] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10606.393470] Lustre: 242941:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10616.761573] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10617.177895] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10626.973205] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10636.602643] LustreError: 244020:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10636.629983] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [10636.632754] LustreError: 244020:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10638.998140] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10640.385611] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [10648.315112] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10660.175022] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10666.201386] Lustre: 244816:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10683.872834] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10683.889196] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10683.923900] Lustre: Skipped 1 previous similar message [10683.940261] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10683.942631] Lustre: Skipped 3 previous similar messages [10686.435848] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10686.449351] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [10686.466805] Lustre: Skipped 2 previous similar messages [10686.491441] Lustre: Skipped 3 previous similar messages [10688.207039] Lustre: server umount mylustre-MDT0000 complete [10691.556577] LustreError: 241797:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10691.594385] LustreError: 241797:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10695.605319] LustreError: 241773:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785398973 with bad export cookie 10056934334345839545 [10695.618791] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10695.630797] LustreError: 241773:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [10696.294543] Lustre: server umount mylustre-MDT0001 complete [10712.225089] Lustre: 235635:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785398974/real 1785398974] req@ffff993affdb5c00 x1872126147744000/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785398990 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10712.326314] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10712.704252] Lustre: server umount mylustre-OST0000 complete [10717.087120] Lustre: 235634:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785398978/real 1785398978] req@ffff9939e8f6d880 x1872126147744256/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785398994 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10718.175814] Lustre: 235636:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785398979/real 1785398979] req@ffff9939cf241500 x1872126147744512/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785398995 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [10719.145994] Lustre: server umount mylustre-OST0001 complete [10748.375209] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [10764.579433] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10765.786898] LustreError: 247131:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10765.965581] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10765.972170] Lustre: Skipped 1 previous similar message [10772.753492] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10784.730529] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10791.384757] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10795.275219] Lustre: 248273:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10804.142194] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10809.904362] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:161) [10811.983659] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [10814.500915] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10827.447575] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10832.958572] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [10842.216303] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10854.630103] Lustre: DEBUG MARKER: Using TIMEOUT=20 [10860.105747] Lustre: 250146:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [10872.289538] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10872.299934] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10872.325144] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [10873.730601] Lustre: server umount tfs-MDT0000 complete [10878.856864] LustreError: 247105:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399156 with bad export cookie 10056934334345841498 [10878.864440] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10878.871906] LustreError: 247105:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [10878.969244] Lustre: tfs-MDT0001: Not available for connect from 0@lo (stopping) [10878.980408] Lustre: Skipped 1 previous similar message [10879.681355] Lustre: server umount tfs-MDT0001 complete [10885.790980] Lustre: server umount tfs-OST0000 complete [10890.915507] Lustre: server umount tfs-OST0001 complete [10921.412510] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [10935.890259] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10936.633607] LustreError: 252397:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10936.662243] LustreError: 252397:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 18 previous similar messages [10936.751262] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10936.763773] Lustre: Skipped 3 previous similar messages [10944.049534] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10960.245806] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10965.885257] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [10969.984257] Lustre: 253541:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [10979.692727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10981.427801] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [10985.561613] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:193) [10989.368198] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11002.311241] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11008.016281] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [11013.785171] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11025.801489] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11030.581213] Lustre: 255414:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11040.040136] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 04:15:16 (1785399316) [11059.178654] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11059.191671] Lustre: Skipped 7 previous similar messages [11059.198884] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11059.206544] Lustre: Skipped 3 previous similar messages [11065.722123] Lustre: server umount lustre-MDT0000 complete [11070.253801] LustreError: 252372:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399348 with bad export cookie 10056934334345843416 [11070.255603] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11070.259622] LustreError: 252372:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [11075.617226] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11075.629861] Lustre: Skipped 7 previous similar messages [11076.948422] Lustre: server umount lustre-MDT0001 complete [11083.163755] Lustre: server umount lustre-OST0000 complete [11088.432469] Lustre: server umount lustre-OST0001 complete [11097.316183] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [11111.206637] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [11159.900648] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [11159.900648] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [11159.900648] [11159.929541] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [11160.234829] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11160.302696] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11160.453317] Lustre: lustre-MDT0000: new disk, initializing [11160.576616] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11160.585901] Lustre: Skipped 3 previous similar messages [11160.614150] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11167.383312] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11184.425781] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [11184.425781] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [11184.425781] [11184.545836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [11184.793361] Lustre: 258615:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11184.916888] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11184.922916] Lustre: Skipped 1 previous similar message [11185.036909] Lustre: lustre-MDT0001: new disk, initializing [11185.405063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11185.438878] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11195.318882] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11213.165369] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11213.443621] Lustre: lustre-OST0000: new disk, initializing [11213.447883] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11213.456262] Lustre: 260200:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11215.338708] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11215.357817] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11215.613745] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11223.402702] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11241.679488] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11241.812232] Lustre: lustre-OST0001: new disk, initializing [11241.816654] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11241.822054] Lustre: 261072:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11243.812977] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11243.843630] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11243.964130] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11251.737211] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11266.735895] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 04:19:03 (1785399543) [11290.725713] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 04:19:27 (1785399567) [11310.571186] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11310.583721] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11310.603786] Lustre: Skipped 3 previous similar messages [11315.706966] LustreError: 260297:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11315.738642] LustreError: 260297:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 22 previous similar messages [11316.012297] Lustre: server umount lustre-OST0000 complete [11323.872179] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11329.560810] Lustre: server umount lustre-MDT0000 complete [11334.661206] LustreError: 259422:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399612 with bad export cookie 10056934334345845194 [11334.665541] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11334.670342] LustreError: 259422:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [11341.284083] Lustre: server umount lustre-MDT0001 complete [11356.314892] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [11373.349865] Lustre: server umount lustre-OST0001 complete [11385.791049] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [11405.980167] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [11465.437844] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11465.796449] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11465.871135] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11466.070322] Lustre: lustre-MDT0000: new disk, initializing [11466.188229] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11466.191203] Lustre: Skipped 3 previous similar messages [11466.221071] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11472.656439] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11489.242496] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11489.632253] Lustre: 265661:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11489.798884] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11489.814660] Lustre: Skipped 1 previous similar message [11489.981366] Lustre: lustre-MDT0001: new disk, initializing [11490.283513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11490.316029] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11498.775921] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11514.547053] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11523.367847] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11532.854280] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11533.105479] Lustre: lustre-OST0000: new disk, initializing [11533.109266] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11533.115374] Lustre: 267632:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11534.469902] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11534.482370] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11534.687571] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11541.549408] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11556.087378] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11570.657835] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11570.683050] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11570.716133] Lustre: Skipped 4 previous similar messages [11570.719163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11570.721467] Lustre: Skipped 7 previous similar messages [11572.338112] Lustre: server umount lustre-OST0000 complete [11583.549122] Lustre: server umount lustre-MDT0000 complete [11588.881590] LustreError: 265655:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785399866 with bad export cookie 10056934334345846657 [11588.905391] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11588.905767] LustreError: 265655:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [11589.030367] LustreError: 235633:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [11590.170455] Lustre: server umount lustre-MDT0001 complete [11617.035465] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [11624.243561] Key type lgssc unregistered [11624.809763] LNet: 269419:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11624.836349] LNetError: 269419:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11624.864248] LNet: Removed LNI 192.168.203.119@tcp [11626.879384] Key type .llcrypt unregistered [11626.881529] Key type ._llcrypt unregistered [11663.628371] Key type ._llcrypt registered [11663.636397] Key type .llcrypt registered [11663.791789] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [11665.874890] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11665.953019] alg: No test for adler32 (adler32-zlib) [11667.120665] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [11667.370861] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [11669.079110] Key type lgssc registered [11670.701751] Lustre: Echo OBD driver; http://www.lustre.org/ [11682.538599] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11682.579332] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11684.179242] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11691.224464] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11693.537793] LustreError: 270885:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11693.558771] LustreError: 270885:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11698.659492] LustreError: 270886:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11703.754967] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11703.777676] LustreError: 270885:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11704.400553] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11711.892603] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11722.085628] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11730.311120] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11739.775883] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11740.522525] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11748.616230] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11753.841671] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [11759.554143] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11761.111245] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [11764.192492] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11764.196246] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11764.209123] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11764.242891] Lustre: Skipped 1 previous similar message [11769.333690] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11769.353391] Lustre: Skipped 2 previous similar messages [11770.141859] Lustre: server umount lustre-OST0000 complete [11775.974195] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11775.986718] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11776.013654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11779.553450] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11779.555935] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11779.590838] Lustre: Skipped 1 previous similar message [11781.864052] Lustre: server umount lustre-MDT0000 complete [11784.671873] LustreError: 270885:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11784.693260] LustreError: 270885:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11786.958824] LustreError: 270866:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785400064 with bad export cookie 17841535370826426211 [11786.963738] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11786.981075] LustreError: 270866:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [11787.867479] Lustre: server umount lustre-MDT0001 complete [11803.953712] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [11807.472581] Key type lgssc unregistered [11807.924492] LNet: 274477:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11807.942152] LNetError: 274477:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11807.973864] LNet: Removed LNI 192.168.203.119@tcp [11809.264899] Key type .llcrypt unregistered [11809.269640] Key type ._llcrypt unregistered [11832.311710] Key type ._llcrypt registered [11832.313681] Key type .llcrypt registered [11832.465322] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [11834.070825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11834.264986] alg: No test for adler32 (adler32-zlib) [11835.379403] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [11835.590815] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [11837.247190] Key type lgssc registered [11839.409462] Lustre: Echo OBD driver; http://www.lustre.org/ [11854.439155] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11854.486106] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11870.239471] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [11875.360119] LustreError: 275674:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.203.119@tcp: failed processing log, type 4: rc = -110 [11902.047189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11913.994139] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [11927.297461] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11930.937169] Lustre: server umount lustre-OST0000 complete [11953.183978] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [11957.914955] Key type lgssc unregistered [11958.464140] LNet: 277307:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11958.474359] LNetError: 277307:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11958.513379] LNet: Removed LNI 192.168.203.119@tcp [11960.225417] Key type .llcrypt unregistered [11960.228151] Key type ._llcrypt unregistered [12022.727860] Key type ._llcrypt registered [12022.747717] Key type .llcrypt registered [12023.130775] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [12025.248703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12025.293695] alg: No test for adler32 (adler32-zlib) [12026.801990] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [12027.381732] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [12029.321535] Key type lgssc registered [12032.379429] Lustre: Echo OBD driver; http://www.lustre.org/ [12061.048314] loop0: detected capacity change from 0 to 204800000 [12061.341041] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12061.362318] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12061.954936] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12063.434639] loop0: detected capacity change from 0 to 204800000 [12063.698044] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12063.741043] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12064.573027] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12066.032322] loop0: detected capacity change from 0 to 204800000 [12066.275781] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12066.285586] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12066.505563] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12067.915097] loop0: detected capacity change from 0 to 204800000 [12068.203986] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12068.224916] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12068.526079] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [12070.021347] loop0: detected capacity change from 0 to 204800000 [12070.052805] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12076.217506] loop0: detected capacity change from 0 to 204800000 [12076.291189] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12081.278084] loop0: detected capacity change from 0 to 204800000 [12081.328718] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12087.911152] loop0: detected capacity change from 0 to 204800000 [12087.998492] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [12094.775414] loop0: detected capacity change from 0 to 204800000 [12094.898219] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12094.956024] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12095.091884] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12095.112847] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [12095.134594] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [12095.147851] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [12095.168563] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [12095.192720] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [12095.214640] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [12095.240794] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [12095.255522] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [12095.269652] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [12095.287784] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [12095.306147] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [12095.340849] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [12095.367937] Lustre: 280730:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [12104.175245] Lustre: server umount MGS complete [12105.787385] loop0: detected capacity change from 0 to 204800000 [12105.960613] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12106.420570] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12106.525649] Lustre: lustre-MDT0000: Aborting client recovery [12107.809687] loop1: detected capacity change from 0 to 204800000 [12110.055525] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12110.109882] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12110.115159] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [12110.124859] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [12110.131975] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [12110.139500] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [12110.144928] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [12110.151375] Lustre: 281162:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [12110.163510] Lustre: Found index 1 for lustre-MDT0001, updating log [12110.317727] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12110.635995] Lustre: lustre-MDT0001: Aborting client recovery [12113.203681] loop2: detected capacity change from 0 to 204800000 [12113.449538] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12113.778075] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12113.811230] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [12113.851082] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [12113.892099] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [12113.926630] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [12113.967921] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [12113.996652] Lustre: 281242:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [12114.380481] Lustre: Found index 0 for lustre-OST0000, updating log [12114.448131] Lustre: 281263:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12114.588091] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12114.631097] Lustre: lustre-OST0000: Aborting recovery [12114.645365] LustreError: 281106:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [12116.580855] LustreError: 281250:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12116.633649] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [12116.637267] LustreError: 281250:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12116.876355] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [12116.989709] loop3: detected capacity change from 0 to 204800000 [12117.226891] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12117.375083] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [12117.403396] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [12117.439244] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [12117.472920] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [12117.501704] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [12117.536289] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [12117.557457] Lustre: 281331:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [12117.602483] Lustre: Found index 1 for lustre-OST0001, updating log [12117.671674] Lustre: 281338:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12117.764498] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12117.786265] Lustre: lustre-OST0001: Aborting recovery [12120.879804] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [12120.880990] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [12120.935394] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [12121.004073] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [12127.262791] LustreError: 281073:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 10190us: rc = -2 [12127.797384] LustreError: 281074:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 511us: rc = -2 [12127.837981] LustreError: 281074:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 14 previous similar messages [12128.868569] LustreError: 281074:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 813us: rc = -2 [12128.913796] LustreError: 281074:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [12146.143903] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12146.150541] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12146.177621] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12146.656227] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12146.683407] Lustre: Skipped 2 previous similar messages [12146.703145] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12146.712115] Lustre: Skipped 2 previous similar messages [12150.320622] Lustre: server umount lustre-MDT0000 complete [12151.780461] LustreError: 281072:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12151.833856] LustreError: 281072:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12152.330733] LustreError: 281057:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785400430 with bad export cookie 2338557145982855095 [12152.362152] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12156.903835] LustreError: 281071:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12156.907130] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12156.910829] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12156.927538] LustreError: 281071:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12156.982268] Lustre: Skipped 1 previous similar message [12159.478213] Lustre: server umount lustre-MDT0001 complete [12167.943124] Lustre: server umount lustre-OST0000 complete [12176.479277] Lustre: server umount lustre-OST0001 complete [12187.350544] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 04:34:23 (1785400463) [12228.281755] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [12244.406199] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [12295.940485] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12296.479868] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12296.560820] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12296.721807] Lustre: lustre-MDT0000: new disk, initializing [12296.910299] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12297.008146] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12304.213294] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12317.990138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12318.203220] Lustre: 285429:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12318.310248] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12318.320907] Lustre: Skipped 1 previous similar message [12318.520360] Lustre: lustre-MDT0001: new disk, initializing [12318.705379] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12318.807148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12318.840585] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12326.260434] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12339.523277] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12351.530313] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12364.300629] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12364.775780] Lustre: lustre-OST0000: new disk, initializing [12364.785775] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12364.795784] Lustre: 287399:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12364.916671] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12373.605265] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12373.631511] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12373.770023] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12374.700704] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12387.721110] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12394.477780] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12405.979182] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12413.399621] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12418.213765] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [12471.273372] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12471.296106] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12471.303332] Lustre: Skipped 1 previous similar message [12471.304437] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12471.304441] Lustre: Skipped 3 previous similar messages [12473.402453] Lustre: server umount lustre-OST0000 complete [12481.505729] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12481.535609] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12481.548590] Lustre: Skipped 1 previous similar message [12485.360131] Lustre: server umount lustre-MDT0000 complete [12486.626225] LustreError: 286848:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12486.659209] LustreError: 286848:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12490.261791] LustreError: 285421:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785400768 with bad export cookie 2338557145982857531 [12490.268331] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12490.281550] LustreError: 285421:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 6 previous similar messages [12491.025966] Lustre: server umount lustre-MDT0001 complete [12502.885118] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12510.081735] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12518.376646] Lustre: server umount MGS complete [12541.152296] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12542.057182] LustreError: 291228:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12542.095253] LustreError: 291241:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [12542.240669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12549.075985] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12551.653884] LustreError: 291229:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12551.679768] LustreError: 291229:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12561.191588] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12568.043761] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12580.016782] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12587.958189] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12598.541549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12608.259889] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12620.095959] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12652.522061] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12652.540979] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12652.562508] Lustre: Skipped 2 previous similar messages [12652.568606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12652.568613] Lustre: Skipped 1 previous similar message [12652.647801] LustreError: Skipped 1 previous similar message [12658.930419] Lustre: server umount lustre-OST0000 complete [12664.290188] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12670.942665] Lustre: server umount lustre-MDT0000 complete [12672.999283] LustreError: 291225:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12673.019238] LustreError: 291225:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12675.726720] LustreError: 291967:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785400953 with bad export cookie 2338557145982859372 [12675.730311] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12675.742804] LustreError: 291967:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [12676.230589] Lustre: server umount lustre-MDT0001 complete [12698.054261] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [12703.442754] Key type lgssc unregistered [12703.723652] LNet: 295200:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12703.737164] LNetError: 295200:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12703.784520] LNet: Removed LNI 192.168.203.119@tcp [12705.469752] Key type .llcrypt unregistered [12705.475488] Key type ._llcrypt unregistered [12763.209188] Key type ._llcrypt registered [12763.212316] Key type .llcrypt registered [12763.433991] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [12784.527229] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [12786.319058] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12786.374906] alg: No test for adler32 (adler32-zlib) [12787.505653] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [12787.778913] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [12789.471136] Key type lgssc registered [12790.913493] Lustre: Echo OBD driver; http://www.lustre.org/ [12845.373200] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12845.417333] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12846.806437] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12846.835604] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12846.918876] Lustre: lustre-MDT0000: new disk, initializing [12847.012079] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12847.059575] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12852.352425] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12866.785852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12866.991720] Lustre: 299165:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12867.063111] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12867.068622] Lustre: Skipped 1 previous similar message [12867.175620] Lustre: lustre-MDT0001: new disk, initializing [12867.330849] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12867.384448] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12867.404425] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12873.952478] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12888.824669] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12896.520021] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12907.583673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12907.928440] Lustre: lustre-OST0000: new disk, initializing [12907.934450] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12907.945192] Lustre: 301137:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12908.058271] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12913.735357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12913.743117] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12913.939157] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12916.096830] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [12932.895634] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12942.943976] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [12953.692351] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [12958.558600] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [12962.653645] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [13006.306456] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13006.335823] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13006.377622] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13011.453918] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13011.462781] Lustre: Skipped 2 previous similar messages [13011.991445] Lustre: server umount lustre-OST0000 complete [13021.153512] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13021.180598] LustreError: Skipped 1 previous similar message [13021.195833] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13021.233667] Lustre: Skipped 1 previous similar message [13021.255149] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13024.059861] Lustre: server umount lustre-MDT0000 complete [13029.985032] LustreError: 302293:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785401307 with bad export cookie 8456736946641563787 [13029.988942] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13030.023548] LustreError: 302293:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [13031.005171] Lustre: server umount lustre-MDT0001 complete [13044.461256] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13050.846764] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [13058.175827] Lustre: server umount MGS complete [13081.409406] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13081.655628] LustreError: 304864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13081.667944] LustreError: 304864:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [13081.679478] LustreError: 304877:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [13081.723650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13087.204218] LustreError: 304865:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13087.704076] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [13091.299574] LustreError: 304864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13096.436952] LustreError: 304865:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13100.771837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13101.619858] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13109.365722] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [13120.866279] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13136.172104] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13153.128169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13153.785178] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13165.047799] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [13174.307741] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13212.130908] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13212.152166] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13212.188072] Lustre: Skipped 1 previous similar message [13212.209200] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13212.223021] Lustre: Skipped 1 previous similar message [13217.232563] Lustre: server umount lustre-OST0000 complete [13223.393674] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13223.426240] Lustre: Skipped 1 previous similar message [13223.445732] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13223.455733] Lustre: Skipped 3 previous similar messages [13228.522572] LustreError: 304864:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13228.831083] Lustre: server umount lustre-MDT0000 complete [13235.378377] LustreError: 304846:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785401513 with bad export cookie 8456736946641565642 [13235.398547] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13235.426325] LustreError: 304846:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 3 previous similar messages [13236.079350] Lustre: server umount lustre-MDT0001 complete [13259.702524] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [13266.598685] Key type lgssc unregistered [13267.210378] LNet: 308842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13267.232273] LNetError: 308842:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13267.285772] LNet: Removed LNI 192.168.203.119@tcp [13269.094381] Key type .llcrypt unregistered [13269.101058] Key type ._llcrypt unregistered [13315.477357] Key type ._llcrypt registered [13315.479708] Key type .llcrypt registered [13315.655779] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [13340.547361] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [13342.308690] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13342.601073] alg: No test for adler32 (adler32-zlib) [13344.419508] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [13344.961484] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [13346.738732] Key type lgssc registered [13348.774973] Lustre: Echo OBD driver; http://www.lustre.org/ [13382.604917] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [13402.559084] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13402.593536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13403.974132] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13404.013665] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13404.150394] Lustre: lustre-MDT0000: new disk, initializing [13404.292527] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13404.323946] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13411.583433] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [13427.338926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13427.512848] Lustre: lustre-OST0000: new disk, initializing [13427.516323] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13427.520729] Lustre: Skipped 1 previous similar message [13427.526310] Lustre: 312903:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13427.588794] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13434.046596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [13434.065863] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [13434.118558] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [13436.488558] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [14725.615065] Lustre: server umount lustre-MDT0000 complete [14732.466086] LustreError: 312086:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403010 with bad export cookie 9779977620327377357 [14732.481901] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14743.382473] Lustre: server umount lustre-OST0000 complete [14788.595957] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [14806.413499] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [14860.036811] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14860.570105] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14860.668925] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14860.857221] Lustre: lustre-MDT0000: new disk, initializing [14861.063707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14861.107047] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14868.404493] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [14883.778310] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14883.939718] Lustre: 317338:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14884.024572] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14884.034307] Lustre: Skipped 1 previous similar message [14884.253131] Lustre: lustre-MDT0001: new disk, initializing [14884.489680] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14884.564037] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14884.580995] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14892.598371] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [14907.748863] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14916.734979] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14927.596796] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14928.170594] Lustre: lustre-OST0000: new disk, initializing [14928.189635] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14928.208717] Lustre: 319312:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14928.495222] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14934.743144] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14934.766658] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14934.983164] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14941.530025] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [14959.028687] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14966.186908] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [14966.526488] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [14973.024021] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [14973.253538] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [14975.967953] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [14975.973736] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14975.987515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14975.995837] Lustre: Skipped 1 previous similar message [14976.010262] Lustre: Skipped 1 previous similar message [14981.102729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14981.112677] Lustre: Skipped 1 previous similar message [14986.213625] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [14986.232425] Lustre: Skipped 1 previous similar message [14990.316484] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14991.001829] Lustre: server umount lustre-OST0000 complete [15001.569564] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15001.603740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15001.612599] Lustre: Skipped 1 previous similar message [15003.609396] Lustre: server umount lustre-MDT0000 complete [15009.849463] LustreError: 317333:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403287 with bad export cookie 9779977620330318197 [15009.863628] LustreError: 317333:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [15009.865402] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15011.156518] Lustre: server umount lustre-MDT0001 complete [15016.492890] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [15019.126377] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 05:21:35 (1785403295) [15029.644682] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15030.345938] LustreError: 321630:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15030.376529] LustreError: 321630:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [15030.437235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15035.890084] LustreError: 321631:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15036.112316] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15040.995898] LustreError: 321630:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15046.112803] LustreError: 321631:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15051.235041] LustreError: 321630:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15051.714576] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15052.966686] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15064.373274] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15075.867075] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15084.881536] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [15098.142773] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15098.898208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15113.716848] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15130.256425] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15140.221921] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15140.485750] Lustre: lustre-OST0001: new disk, initializing [15140.487808] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [15140.505680] Lustre: 324474:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [15140.666364] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15149.180950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [15149.207933] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [15149.296347] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [15149.700915] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15150.390840] LustreError: 324805:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [15150.405570] LustreError: 324805:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [15151.395796] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [15153.459414] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [15153.487438] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [15157.601979] LustreError: 324805:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [15157.620820] LustreError: 324805:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [15163.613747] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [15163.921358] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [15165.802745] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [15165.821144] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [15172.548488] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff92a3b2d9a800.ost_server_uuid 50 [15174.916710] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff92a3b2d9a800.ost_server_uuid in FULL state after 0 sec [15182.243407] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff92a3b2d9a800.ost_server_uuid 50 [15182.435255] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [15182.449027] LustreError: 324794:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 31 previous similar messages [15184.404410] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff92a3b2d9a800.ost_server_uuid in IDLE state after 0 sec [15186.671676] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [15202.272402] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [15202.275556] LustreError: Skipped 1 previous similar message [15202.285491] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15202.307788] Lustre: Skipped 1 previous similar message [15202.315834] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [15208.229715] Lustre: server umount lustre-OST0001 complete [15210.999036] LustreError: 324796:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15211.027199] LustreError: 324796:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [15218.660725] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [15218.670861] LustreError: Skipped 1 previous similar message [15218.679515] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15218.701337] Lustre: Skipped 1 previous similar message [15218.713836] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15218.721987] Lustre: Skipped 5 previous similar messages [15224.317272] Lustre: server umount lustre-OST0000 complete [15231.458792] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15231.480558] Lustre: Skipped 2 previous similar messages [15236.451652] Lustre: server umount lustre-MDT0000 complete [15236.587754] LustreError: 321631:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15236.628381] LustreError: 321631:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [15244.088994] LustreError: 321610:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403521 with bad export cookie 9779977620330319079 [15244.090537] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15244.111753] LustreError: 321610:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [15245.744375] Lustre: server umount lustre-MDT0001 complete [15266.858888] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [15270.637907] Key type lgssc unregistered [15271.050475] LNet: 327184:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15271.060353] LNetError: 327184:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15271.084432] LNet: Removed LNI 192.168.203.119@tcp [15271.965199] Key type .llcrypt unregistered [15271.966727] Key type ._llcrypt unregistered [15304.311513] Key type ._llcrypt registered [15304.315287] Key type .llcrypt registered [15304.538613] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [15306.913447] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [15306.967300] alg: No test for adler32 (adler32-zlib) [15308.395542] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [15308.850336] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [15310.583152] Key type lgssc registered [15312.726251] Lustre: Echo OBD driver; http://www.lustre.org/ [15324.724432] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [15324.766925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15326.570993] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15333.370641] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15336.943711] LustreError: 328608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15336.975622] LustreError: 328608:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15342.049430] LustreError: 328609:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15346.028590] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [15347.171958] LustreError: 328608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15351.274619] LustreError: 328609:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15356.388526] LustreError: 328608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15356.578222] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15357.407667] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15367.159956] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15387.472936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15388.055829] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15396.229286] LustreError: 330293:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15396.243798] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [15398.318868] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15409.573276] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15412.720871] LustreError: 330294:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15412.789125] LustreError: 330294:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [15424.558728] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15424.713863] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15430.195310] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [15439.833437] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15452.326404] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [15462.150859] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [15462.381891] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [15583.718308] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [15583.739886] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15583.764202] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [15583.795778] Lustre: Skipped 1 previous similar message [15588.358243] Lustre: server umount lustre-OST0001 complete [15588.835208] LustreError: 331914:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15588.901856] LustreError: 331914:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15604.213448] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15604.243299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15604.259568] Lustre: Skipped 2 previous similar messages [15604.299760] Lustre: Skipped 1 previous similar message [15609.313351] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [15609.325303] Lustre: Skipped 1 previous similar message [15610.253456] Lustre: server umount lustre-OST0000 complete [15619.041153] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [15619.055792] LustreError: Skipped 1 previous similar message [15619.067645] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15619.102979] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15622.117419] Lustre: server umount lustre-MDT0000 complete [15628.131426] LustreError: 328588:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785403905 with bad export cookie 6246582239393355120 [15628.158802] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15628.161399] LustreError: 328588:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [15628.873679] Lustre: server umount lustre-MDT0001 complete [15648.789269] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [15656.077370] Key type lgssc unregistered [15656.548325] LNet: 333586:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15656.556441] LNetError: 333586:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15657.573648] LNet: Removed LNI 192.168.203.119@tcp [15659.088381] Key type .llcrypt unregistered [15659.093315] Key type ._llcrypt unregistered [15718.121298] Key type ._llcrypt registered [15718.124496] Key type .llcrypt registered [15718.260473] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [15719.592920] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [15719.801049] alg: No test for adler32 (adler32-zlib) [15721.160976] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [15721.524435] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [15723.271355] Key type lgssc registered [15725.124445] Lustre: Echo OBD driver; http://www.lustre.org/ [15740.499736] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [15740.536586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15742.888421] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15751.172978] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15753.199274] LustreError: 335881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15753.222317] LustreError: 335881:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [15758.345791] LustreError: 335882:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15763.441875] LustreError: 335881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15768.570786] LustreError: 335882:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15773.536531] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15773.669531] LustreError: 335881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15775.150584] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15784.527691] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15788.290812] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [15798.696119] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15799.456576] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15800.488650] LustreError: 337374:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15808.113859] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15814.145249] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [15819.252090] LustreError: 337429:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15819.287946] LustreError: 337429:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [15820.090686] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15820.386368] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [15825.417708] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [15828.921516] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15840.851853] Lustre: DEBUG MARKER: Using TIMEOUT=20 [15850.168321] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [15871.462896] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15871.468503] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [15871.491958] Lustre: Skipped 2 previous similar messages [15871.510732] Lustre: Skipped 3 previous similar messages [15876.660832] LustreError: 335881:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15877.032647] Lustre: server umount lustre-MDT0000 complete [15882.479919] LustreError: 337398:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404160 with bad export cookie 15535067789052997527 [15882.492019] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [15882.503393] LustreError: 337398:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [15883.245553] Lustre: server umount lustre-MDT0001 complete [15899.752684] Lustre: server umount lustre-OST0000 complete [15902.687662] Lustre: 335143:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785404164/real 1785404164] req@ffff993ac5a4ed80 x1872131783734016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785404180 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15902.744324] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [15902.774356] Lustre: Skipped 1 previous similar message [15903.201664] Lustre: 335145:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785404165/real 1785404165] req@ffff993af4f46300 x1872131783734272/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785404181 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [15906.790764] Lustre: server umount lustre-OST0001 complete [15936.432689] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [15951.839895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15952.664996] LustreError: 341157:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [15952.699259] LustreError: 341157:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [15952.841777] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [15959.825112] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15971.126857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [15971.547303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [15977.005379] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [15981.214154] Lustre: 342298:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [15990.855348] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [15991.519135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [15997.694498] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [16000.564540] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16013.759853] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16019.492642] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [16025.304464] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16039.686584] Lustre: DEBUG MARKER: Using TIMEOUT=20 [16045.444363] Lustre: 344174:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [16053.735201] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [16053.747847] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16053.774298] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16055.270819] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16055.282434] Lustre: Skipped 2 previous similar messages [16057.965787] Lustre: server umount lustre-MDT0000 complete [16063.800751] LustreError: 341139:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404341 with bad export cookie 15535067789052999228 [16063.802208] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16063.812355] LustreError: 341139:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [16064.571976] Lustre: server umount lustre-MDT0001 complete [16080.110568] Lustre: server umount lustre-OST0000 complete [16081.631958] Lustre: 335143:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785404343/real 1785404343] req@ffff9939d72ff800 x1872131783795968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785404359 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [16081.677802] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16081.706697] Lustre: Skipped 3 previous similar messages [16084.706199] Lustre: 335144:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1785404346/real 1785404346] req@ffff993af61da300 x1872131783796224/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1785404362 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [16085.648781] Lustre: server umount lustre-OST0001 complete [16090.163266] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [16092.594405] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [16094.959465] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:39:31 (1785404371) [16126.968011] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [16146.279718] loop0: detected capacity change from 0 to 1099511627776 [16148.274907] SGI XFS with ACLs, security attributes, quota, no debug enabled [16148.298973] *** VALIDATE xfs *** [16148.340600] XFS (loop0): Mounting V5 Filesystem [16148.447318] XFS (loop0): Ending clean mount [16151.258682] loop1: detected capacity change from 0 to 18691697672192 [16328.212185] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [16344.168636] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [16398.656114] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16398.949241] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [16398.981834] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [16399.101946] Lustre: lustre-MDT0000: new disk, initializing [16399.190894] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16399.195959] Lustre: Skipped 1 previous similar message [16399.219328] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [16404.748087] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16419.320726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16419.548774] Lustre: 349359:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16419.649580] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [16419.656429] Lustre: Skipped 1 previous similar message [16419.832463] Lustre: lustre-MDT0001: new disk, initializing [16420.130306] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [16420.151070] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [16428.568830] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16442.837386] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16450.735781] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16460.615580] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16461.001865] Lustre: lustre-OST0000: new disk, initializing [16461.006081] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [16461.019862] Lustre: 351332:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [16462.555171] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [16462.575597] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [16462.769599] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [16471.081356] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16484.252672] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16491.402624] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [16491.674654] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [16497.105584] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [16497.499266] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [16503.776566] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [16503.796610] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16503.812209] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [16503.820846] Lustre: Skipped 1 previous similar message [16506.452580] Lustre: server umount lustre-OST0000 complete [16514.024350] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16514.051824] Lustre: Skipped 1 previous similar message [16514.063783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [16514.077077] Lustre: Skipped 1 previous similar message [16520.295834] Lustre: server umount lustre-MDT0000 complete [16523.244761] LustreError: 349370:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16523.273437] LustreError: 349370:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 17 previous similar messages [16526.484157] LustreError: 349350:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404804 with bad export cookie 15535067789053000782 [16526.492499] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16526.520721] LustreError: 349350:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 1 previous similar message [16527.688739] Lustre: server umount lustre-MDT0001 complete [16540.640845] XFS (loop0): Unmounting Filesystem [16547.814933] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 05:47:04 (1785404824) [16555.519945] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16556.027286] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16556.029947] Lustre: Skipped 2 previous similar messages [16562.401421] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16576.654646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16586.191451] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16597.475218] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16606.018389] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16619.624477] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16631.170088] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16645.905652] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16661.473794] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16661.489118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [16661.491532] Lustre: Skipped 2 previous similar messages [16661.516648] Lustre: Skipped 3 previous similar messages [16665.380828] Lustre: server umount lustre-OST0000 complete [16677.649855] Lustre: server umount lustre-MDT0000 complete [16683.760604] LustreError: 353872:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785404961 with bad export cookie 15535067789053001664 [16683.774945] LustreError: 353872:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 2 previous similar messages [16683.778910] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [16685.079223] Lustre: server umount lustre-MDT0001 complete [16712.144438] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [16716.616489] Key type lgssc unregistered [16717.160296] LNet: 357552:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [16717.169501] LNetError: 357552:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [16718.246945] LNet: Removed LNI 192.168.203.119@tcp [16719.185135] Key type .llcrypt unregistered [16719.188353] Key type ._llcrypt unregistered [16755.071697] Key type ._llcrypt registered [16755.079104] Key type .llcrypt registered [16755.264439] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [16757.320543] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [16757.924707] alg: No test for adler32 (adler32-zlib) [16759.101672] Lustre: Lustre: Build Version: 2.17.56_2_g669939b [16759.347488] LNet: Added LNI 192.168.203.119@tcp [8/256/0/180] [16761.001692] Key type lgssc registered [16763.043900] Lustre: Echo OBD driver; http://www.lustre.org/ [16774.642962] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [16774.683375] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16776.366803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [16782.316573] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16786.940079] LustreError: 359126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16786.959460] LustreError: 359126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16792.040928] LustreError: 359125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16794.394843] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16794.709142] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16800.488624] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16812.198844] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [16821.926139] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [16833.937395] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [16834.345324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [16843.682565] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16855.274411] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [16866.272567] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16866.281553] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16866.325377] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16867.298611] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16867.301504] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16872.430601] LustreError: 359126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16872.470262] LustreError: 359126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [16872.566275] Lustre: server umount lustre-MDT0001 complete [16877.542199] LustreError: 359125:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16877.568781] LustreError: 359125:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16882.658982] LustreError: 359120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16882.721737] LustreError: 359120:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [16892.051621] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16892.284513] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16892.292752] Lustre: Found index 1 for lustre-MDT0001, updating log [16892.317131] Lustre: 359114:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16892.815839] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16898.063567] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16899.636794] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [16902.130798] LustreError: 362652:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16902.143616] LustreError: 362652:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16902.163376] LustreError: 359117:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16902.254587] LustreError: 362659:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.119@tcp: cfg command failed: rc = -17 [16902.263328] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [16902.263328] [16923.652508] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [16923.658621] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [16923.670217] Lustre: Skipped 1 previous similar message [16923.670669] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [16923.713068] Lustre: Skipped 3 previous similar messages [16927.212433] Lustre: server umount lustre-MDT0001 complete [16928.746303] LustreError: 360568:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [16928.782498] LustreError: 360568:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [16947.308693] LustreError: 359120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [16947.343913] LustreError: 359120:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 10 previous similar messages [16951.570617] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [16952.125040] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [16952.137863] Lustre: Found index 1 for lustre-MDT0001, updating log [16952.273603] Lustre: 359115:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [16953.303065] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [16958.469593] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [16960.506089] LustreError: 363598:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [16960.518325] LustreError: 363598:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [16960.536592] LustreError: 359117:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [16960.546039] LustreError: 359117:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [16960.693714] LustreError: 363603:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [16960.706914] LustreError: 363603:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [16960.720783] LustreError: 363603:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.119@tcp: cfg command failed: rc = -17 [16960.739144] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.203.119@tcp [16960.739144] [16961.752821] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [17269.852898] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.19@tcp (stopping) [17269.861513] Lustre: Skipped 1 previous similar message [17270.752689] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [17270.765672] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17270.848714] Lustre: Skipped 1 previous similar message [17274.926519] Lustre: lustre-MDT0001: Not available for connect from 192.168.203.19@tcp (stopping) [17274.933398] Lustre: Skipped 2 previous similar messages [17276.211279] Lustre: server umount lustre-MDT0001 complete [17280.060766] LustreError: 360568:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.203.19@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [17280.102175] LustreError: 360568:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [17302.020886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17302.292199] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [17302.302220] Lustre: Found index 1 for lustre-MDT0001, updating log [17302.385592] Lustre: 359114:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [17303.118266] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [17308.156659] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [17308.772521] LustreError: 370487:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [17308.808183] LustreError: 370487:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [17308.878321] LustreError: 359117:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [17308.908986] LustreError: 359117:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [17309.093863] LustreError: 370490:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [17309.175995] LustreError: 370490:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.203.119@tcp: cfg command failed: rc = -114 [17309.191438] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [17309.191438] [17313.127395] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [17640.929910] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [17640.944675] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17640.951681] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [17640.951689] Lustre: Skipped 2 previous similar messages [17640.991990] Lustre: Skipped 2 previous similar messages [17644.793528] Lustre: server umount lustre-MDT0000 complete [17646.062539] LustreError: 359121:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [17646.111593] LustreError: 359121:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [17649.978079] LustreError: 359105:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785405927 with bad export cookie 17573066249125143297 [17649.984045] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [17650.001907] LustreError: 359105:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) Skipped 4 previous similar messages [17651.168349] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17656.717840] Lustre: server umount lustre-MDT0001 complete [17665.526568] Lustre: server umount lustre-OST0000 complete [17682.234145] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [17696.975458] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [17753.120623] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:07:09 (1785406029) [17763.254498] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17763.850920] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [17763.941438] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [17764.177922] Lustre: lustre-MDT0000: new disk, initializing [17764.312031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [17764.345243] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [17771.379447] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [17788.884726] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [17789.268800] Lustre: 380100:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [17789.407546] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [17789.415719] Lustre: Skipped 1 previous similar message [17789.600044] Lustre: lustre-MDT0001: new disk, initializing [17789.856296] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [17789.944444] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [17789.976217] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [17798.410530] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [17812.622284] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [17821.655437] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [17841.001467] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [17841.699167] Lustre: lustre-OST0000: new disk, initializing [17841.713921] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [17841.741705] Lustre: 382070:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [17842.077419] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [17848.003584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [17848.033828] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [17848.322599] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [17856.443398] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_default_debug -1 all [17872.446854] Lustre: DEBUG MARKER: oleg319-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [17887.210229] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [17887.219580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [17887.237477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [17887.245049] Lustre: Skipped 4 previous similar messages [17889.265706] LustreError: 380111:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [17889.303848] LustreError: 380111:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [17890.341418] Lustre: server umount lustre-MDT0000 complete [17897.354140] LustreError: 380090:0:(ldlm_lockd.c:2567:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1785406175 with bad export cookie 17573066249125145250 [17897.374789] LustreError: MGC192.168.203.119@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [17898.308822] Lustre: server umount lustre-MDT0001 complete [17915.501727] Lustre: server umount lustre-OST0000 complete [17958.306446] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing set_hostid [17982.498144] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing load_modules_local [18030.010923] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 17556 sec ========= 06:11:45 (1785406305) [18033.257568] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:11:48 (1785406308) === [18041.032679] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:11:56 (1785406316) === [18104.468680] Lustre: DEBUG MARKER: oleg319-server.virtnet: executing unload_modules_local [18109.333554] Key type lgssc unregistered [18109.848625] LNet: 388473:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [18109.857946] LNetError: 388473:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [18109.873385] LNet: Removed LNI 192.168.203.119@tcp [18111.286535] Key type .llcrypt unregistered [18111.291126] Key type ._llcrypt unregistered