[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 486369341 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524584K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003351] x2apic enabled [ 0.004011] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008698] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.009024] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.010015] pid_max: default: 32768 minimum: 301 [ 0.012046] LSM: Security Framework initializing [ 0.013000] Yama: becoming mindful. [ 0.013041] SELinux: Initializing. [ 0.014070] *** VALIDATE selinux *** [ 0.022841] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027253] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028168] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029124] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030120] *** VALIDATE tmpfs *** [ 0.031443] *** VALIDATE proc *** [ 0.033088] *** VALIDATE cgroup *** [ 0.033918] *** VALIDATE cgroup2 *** [ 0.035155] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.036162] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.037012] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.038023] Spectre V2 : User space: Vulnerable [ 0.039008] Speculative Store Bypass: Vulnerable [ 0.041104] debug: unmapping init [mem 0xffffffff9aa59000-0xffffffff9aa60fff] [ 0.043241] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.044831] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.045021] ... version: 2 [ 0.045943] ... bit width: 48 [ 0.046011] ... generic registers: 4 [ 0.047009] ... value mask: 0000ffffffffffff [ 0.048011] ... max period: 00007fffffffffff [ 0.049007] ... fixed-purpose events: 3 [ 0.049830] ... event mask: 000000070000000f [ 0.050239] rcu: Hierarchical SRCU implementation. [ 0.052534] smp: Bringing up secondary CPUs ... [ 0.053440] x86: Booting SMP configuration: [ 0.054015] .... node #0, CPUs: #1 #2 #3 [ 0.062164] smp: Brought up 1 node, 4 CPUs [ 0.064012] smpboot: Max logical packages: 1 [ 0.065013] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.123021] node 0 deferred pages initialised in 54ms [ 0.126119] devtmpfs: initialized [ 0.127227] x86/mm: Memory block size: 128MB [ 0.129894] gcov: version magic: 0x41383552 [ 0.130646] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.134084] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.136309] pinctrl core: initialized pinctrl subsystem [ 0.138168] [ 0.138584] ************************************************************* [ 0.141015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.143013] ** ** [ 0.145013] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.147011] ** ** [ 0.150014] ** This means that this kernel is built to expose internal ** [ 0.152012] ** IOMMU data structures, which may compromise security on ** [ 0.154021] ** your system. ** [ 0.156016] ** ** [ 0.158014] ** If you see this message and you are not debugging the ** [ 0.160012] ** kernel, report this immediately to your vendor! ** [ 0.162011] ** ** [ 0.164013] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.166013] ************************************************************* [ 0.167690] NET: Registered protocol family 16 [ 0.169428] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.172072] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.175063] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.179010] cpuidle: using governor menu [ 0.180560] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.182373] PCI: Using configuration type 1 for base access [ 0.185131] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.196054] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.198072] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.202061] cryptd: max_cpu_qlen set to 1000 [ 0.204233] ACPI: Added _OSI(Module Device) [ 0.205000] ACPI: Added _OSI(Processor Device) [ 0.207022] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.209015] ACPI: Added _OSI(Processor Aggregator Device) [ 0.213368] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.219486] ACPI: Interpreter enabled [ 0.220081] ACPI: PM: (supports S0 S3 S4 S5) [ 0.221013] ACPI: Using IOAPIC for interrupt routing [ 0.223104] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.226453] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.237000] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.239044] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.242017] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.246100] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.252394] acpiphp: Slot [2] registered [ 0.253139] acpiphp: Slot [5] registered [ 0.255108] acpiphp: Slot [6] registered [ 0.256111] acpiphp: Slot [7] registered [ 0.258125] acpiphp: Slot [8] registered [ 0.259251] acpiphp: Slot [9] registered [ 0.261131] acpiphp: Slot [10] registered [ 0.263129] acpiphp: Slot [3] registered [ 0.264121] acpiphp: Slot [4] registered [ 0.266108] acpiphp: Slot [11] registered [ 0.268131] acpiphp: Slot [12] registered [ 0.269094] acpiphp: Slot [13] registered [ 0.271108] acpiphp: Slot [14] registered [ 0.272098] acpiphp: Slot [15] registered [ 0.274117] acpiphp: Slot [16] registered [ 0.275091] acpiphp: Slot [17] registered [ 0.277122] acpiphp: Slot [18] registered [ 0.278098] acpiphp: Slot [19] registered [ 0.280092] acpiphp: Slot [20] registered [ 0.281088] acpiphp: Slot [21] registered [ 0.283099] acpiphp: Slot [22] registered [ 0.284098] acpiphp: Slot [23] registered [ 0.286092] acpiphp: Slot [24] registered [ 0.287135] acpiphp: Slot [25] registered [ 0.289105] acpiphp: Slot [26] registered [ 0.291107] acpiphp: Slot [27] registered [ 0.292161] acpiphp: Slot [28] registered [ 0.294120] acpiphp: Slot [29] registered [ 0.295097] acpiphp: Slot [30] registered [ 0.297116] acpiphp: Slot [31] registered [ 0.298085] PCI host bridge to bus 0000:00 [ 0.300019] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.302031] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.304020] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.306026] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.309031] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.312047] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.314240] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.317180] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.321201] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.330013] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.335116] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.337017] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.340016] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.342020] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.345401] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.347784] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.350041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.352837] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.358014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.374021] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.379014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.384347] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.392017] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.401017] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.422018] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.433567] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.441016] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.448016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.468036] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.480923] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.490027] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.496019] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.519021] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.528866] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.537033] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.544025] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.565035] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.577548] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.585020] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.592020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.610031] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.620830] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.635026] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.640022] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.658029] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.669821] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.672424] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.674402] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.677477] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.680420] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.684165] iommu: Default domain type: Passthrough [ 0.687464] SCSI subsystem initialized [ 0.689178] ACPI: bus type USB registered [ 0.690149] usbcore: registered new interface driver usbfs [ 0.692133] usbcore: registered new interface driver hub [ 0.694119] usbcore: registered new device driver usb [ 0.696258] pps_core: LinuxPPS API ver. 1 registered [ 0.698018] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.701093] PTP clock support registered [ 0.703180] EDAC MC: Ver: 3.0.0 [ 0.705136] PCI: Using ACPI for IRQ routing [ 0.708047] NetLabel: Initializing [ 0.709019] NetLabel: domain hash size = 128 [ 0.710018] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.713119] NetLabel: unlabeled traffic allowed by default [ 0.716059] vgaarb: loaded [ 0.717351] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.719034] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.728216] clocksource: Switched to clocksource kvm-clock [ 0.842527] VFS: Disk quotas dquot_6.6.0 [ 0.844304] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.847322] *** VALIDATE ramfs *** [ 0.848769] *** VALIDATE hugetlbfs *** [ 0.850644] pnp: PnP ACPI init [ 0.853404] pnp: PnP ACPI: found 6 devices [ 0.871978] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.874463] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.875974] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.877231] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.878689] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.880568] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.883072] NET: Registered protocol family 2 [ 0.885486] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.890366] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.894800] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.901642] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.904754] TCP: Hash tables configured (established 65536 bind 65536) [ 0.907707] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.910508] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.913063] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.915478] NET: Registered protocol family 1 [ 0.918094] RPC: Registered named UNIX socket transport module. [ 0.919576] RPC: Registered udp transport module. [ 0.921021] RPC: Registered tcp transport module. [ 0.922437] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.924784] NET: Registered protocol family 44 [ 0.926220] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.928065] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.929903] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.932316] PCI: CLS 0 bytes, default 64 [ 0.934300] Unpacking initramfs... [ 2.416614] debug: unmapping init [mem 0xffff95f8bcc54000-0xffff95f8bffbffff] [ 2.423118] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.425479] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.429782] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 2.921667] Initialise system trusted keyrings [ 2.923342] Key type blacklist registered [ 2.925149] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.935680] zbud: loaded [ 2.939283] *** VALIDATE nfs *** [ 2.939960] *** VALIDATE nfs4 *** [ 2.940929] pstore: using deflate compression [ 2.944399] Platform Keyring initialized [ 3.080994] NET: Registered protocol family 38 [ 3.082802] Key type asymmetric registered [ 3.083815] Asymmetric key parser 'x509' registered [ 3.085798] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.088196] io scheduler mq-deadline registered [ 3.089443] io scheduler kyber registered [ 3.090582] io scheduler bfq registered [ 3.091802] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.093834] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.095782] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.098121] ACPI: Power Button [PWRF] [ 3.104609] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.111292] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.121238] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.126831] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.137645] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.163054] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.190043] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.193905] Non-volatile memory driver v1.3 [ 3.194993] Linux agpgart interface v0.103 [ 3.227310] virtio_blk virtio1: [vda] 147136 512-byte logical blocks (75.3 MB/71.8 MiB) [ 3.231934] vda: detected capacity change from 0 to 75333632 [ 3.248507] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.250244] vdb: detected capacity change from 0 to 1073741824 [ 3.267664] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.270616] vdc: detected capacity change from 0 to 2621440000 [ 3.286282] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.289865] vdd: detected capacity change from 0 to 2621440000 [ 3.320158] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.322831] vde: detected capacity change from 0 to 4294967296 [ 3.339338] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.342159] vdf: detected capacity change from 0 to 4294967296 [ 3.352682] libphy: Fixed MDIO Bus: probed [ 3.359934] usbcore: registered new interface driver usbserial_generic [ 3.362475] usbserial: USB Serial support registered for generic [ 3.364733] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.369024] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.370568] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.373241] mousedev: PS/2 mouse device common for all mice [ 3.377245] rtc_cmos 00:05: RTC can wake from S4 [ 3.381744] rtc_cmos 00:05: registered as rtc0 [ 3.384677] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.385158] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.390789] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.390972] intel_pstate: CPU model not supported [ 3.395382] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.400143] hid: raw HID events driver (C) Jiri Kosina [ 3.402424] usbcore: registered new interface driver usbhid [ 3.404637] usbhid: USB HID core driver [ 3.406351] drop_monitor: Initializing network drop monitor service [ 3.408842] Initializing XFRM netlink socket [ 3.410715] NET: Registered protocol family 10 [ 3.413502] Segment Routing with IPv6 [ 3.414786] NET: Registered protocol family 17 [ 3.416645] mpls_gso: MPLS GSO support [ 3.420949] RAS: Correctable Errors collector initialized. [ 3.422407] AVX version of gcm_enc/dec engaged. [ 3.423694] AES CTR mode by8 optimization enabled [ 3.491844] sched_clock: Marking stable (3491811372, 0)->(4422114622, -930303250) [ 3.494778] registered taskstats version 1 [ 3.496438] Loading compiled-in X.509 certificates [ 3.498046] zswap: loaded using pool lzo/zbud [ 3.523330] Key type big_key registered [ 3.538435] Key type encrypted registered [ 3.540234] ima: No TPM chip found, activating TPM-bypass! [ 3.542651] ima: Allocated hash algorithm: sha1 [ 3.544085] ima: No architecture policies found [ 3.545346] evm: Initialising EVM extended attributes: [ 3.546986] evm: security.selinux [ 3.548246] evm: security.ima [ 3.549327] evm: security.capability [ 3.550433] evm: HMAC attrs: 0x1 [ 3.553164] rtc_cmos 00:05: setting system clock to 2026-09-11 14:06:41 UTC (1789135601) [ 3.560130] debug: unmapping init [mem 0xffffffff9ba03000-0xffffffff9bbfffff] [ 3.563515] debug: unmapping init [mem 0xffffffff9a782000-0xffffffff9aa58fff] [ 3.568100] Write protecting the kernel read-only data: 28672k [ 3.570544] debug: unmapping init [mem 0xffffffff98e03000-0xffffffff98ffffff] [ 3.572712] debug: unmapping init [mem 0xffffffff99714000-0xffffffff997fffff] [ 3.609840] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.616630] systemd[1]: Detected virtualization kvm. [ 3.617934] systemd[1]: Detected architecture x86-64. [ 3.619384] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.645362] systemd[1]: No hostname configured. [ 3.646538] systemd[1]: Set hostname to . [ 3.647921] random: systemd: uninitialized urandom read (16 bytes read) [ 3.649743] systemd[1]: Initializing machine ID from random generator. [ 3.787249] random: systemd: uninitialized urandom read (16 bytes read) [ 3.790033] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 3.793682] random: systemd: uninitialized urandom read (16 bytes read) [ 3.796595] systemd[1]: Reached target Swap. [ OK ] Reached target Swap. [ 3.800340] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on Journal Socket. Starting Create list of required st…ce nodes for the current kernel... Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Apply Kernel Variables... [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. Starting dracut cmdline hook... Starting Create Static Device Nodes in /dev... [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.448831] device-mapper: uevent: version 1.0.3 [ 4.451394] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.198981] random: fast init done [ 5.216770] virtio_net virtio0 ens2: renamed from eth0 [ 5.265608] scsi host0: ata_piix [ 5.283248] scsi host1: ata_piix [ 5.284905] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.286881] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.795440] dracut-initqueue[593]: RTNETLINK answers: File exists [ 10.068685] random: crng init done [ 10.070330] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.493890] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Timers. [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped target Slices. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 11.646046] printk: systemd: 26 output lines suppressed due to ratelimiting [ 11.852518] SELinux: Disabled at runtime. [ 11.911551] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 11.921561] systemd[1]: Detected virtualization kvm. [ 11.924172] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 12.415516] systemd[1]: initrd-switch-root.service: Succeeded. [ 12.418710] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 12.425338] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 12.428641] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 12.431703] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 12.439676] systemd[1]: Starting Journal Service... Starting Journal Service... [ 12.450115] systemd[1]: Mounting Kernel Debug File System... Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Listening on Process Core Dump Socket. [ OK ] Created slice system-serial\x2dgetty.slice. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice system-sshd\x2dkeygen.slice. [ OK ] Listening on udev Control Socket. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Forward Password Requests to Wall Directory Watch. Starting udev Coldplug all Devices... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Mounting Huge Pages File System... Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Initrd File Systems. [ 12.617437] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Starting Apply Kernel Variables... Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started udev Coldplug all Devices. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 12.994955] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.329223] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 13.390391] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.474226] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 13.499092] EDAC sbridge: Ver: 1.1.2 [ 14.899126] Key type dns_resolver registered [ 15.201700] NFS: Registering the id_resolver key type [ 15.203726] Key type id_resolver registered [ 15.205325] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started dnf makecache --timer. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Network Manager... [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... Starting Hostname Service... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting System Logging Service... Starting Notify NFS peers of a restart... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg146-server login: [ 44.609700] libcfs: loading out-of-tree module taints kernel. [ 44.629944] Key type ._llcrypt registered [ 44.631683] Key type .llcrypt registered [ 44.693909] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 53.638453] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 54.436282] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 54.445997] alg: No test for adler32 (adler32-zlib) [ 55.492140] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 55.848409] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 57.480303] Key type lgssc registered [ 59.405410] Lustre: Echo OBD driver; http://www.lustre.org/ [ 76.362694] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 83.734046] hrtimer: interrupt took 4242383 ns [ 125.909262] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 139.957795] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 139.991272] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 141.263189] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 141.297370] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 141.444288] Lustre: lustre-MDT0000: new disk, initializing [ 141.553941] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 141.569950] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 146.256749] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 161.332107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 161.503716] Lustre: 6491:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 161.541606] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 161.546861] Lustre: Skipped 1 previous similar message [ 161.632811] Lustre: lustre-MDT0001: new disk, initializing [ 161.718865] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 161.742060] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 161.755736] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 167.361836] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 172.382968] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 183.439212] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 183.799778] Lustre: lustre-OST0000: new disk, initializing [ 183.805989] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 183.826772] Lustre: 8427:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 183.897464] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 186.903566] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 186.927983] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 187.035937] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 191.473645] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 207.764039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 207.873667] Lustre: lustre-OST0001: new disk, initializing [ 207.877728] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 207.883900] Lustre: 9497:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 207.965357] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 213.549640] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 213.559781] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 213.663986] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 214.855329] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 227.937436] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 235.643352] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 242.882178] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing check_logdir /tmp/testlogs/ [ 247.483293] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing yml_node [ 254.261900] Lustre: DEBUG MARKER: Client: 2.17.57.127 [ 257.889089] Lustre: DEBUG MARKER: MDS: 2.17.57.127 [ 262.730595] Lustre: DEBUG MARKER: OSS: 2.17.57.127 [ 265.740709] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Sep 11 10:11:01 EDT 2026 [ 288.165510] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 289.925975] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 306.145338] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 306.152235] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 306.156519] Lustre: Skipped 3 previous similar messages [ 306.171403] Lustre: Skipped 3 previous similar messages [ 311.265583] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 311.277026] Lustre: Skipped 3 previous similar messages [ 311.819815] Lustre: server umount lustre-MDT0000 complete [ 316.081044] LustreError: 9509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789135914 with bad export cookie 10341427104508513799 [ 316.082984] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 316.092090] LustreError: 9509:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 316.385596] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.406158] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 316.415473] Lustre: Skipped 1 previous similar message [ 321.505611] LustreError: 10002:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 321.506620] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 321.532486] LustreError: 10002:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 321.539937] Lustre: Skipped 1 previous similar message [ 322.508075] Lustre: server umount lustre-MDT0001 complete [ 326.950651] Lustre: server umount lustre-OST0000 complete [ 331.245560] Lustre: server umount lustre-OST0001 complete [ 338.627331] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 348.675942] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 390.980810] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 391.425271] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 391.489843] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 391.696531] Lustre: lustre-MDT0000: new disk, initializing [ 391.863073] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 391.889327] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 397.495408] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 410.421577] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 410.610392] Lustre: 15058:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 410.696700] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 410.702505] Lustre: Skipped 1 previous similar message [ 410.811512] Lustre: lustre-MDT0001: new disk, initializing [ 410.942449] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 411.022601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 411.037078] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 415.771142] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 428.408174] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 435.824759] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 443.862043] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 444.113415] Lustre: lustre-OST0000: new disk, initializing [ 444.118924] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 444.124613] Lustre: 17027:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 444.243252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 446.021571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 446.035507] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 446.267206] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 451.400484] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 462.311839] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 468.073154] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 468.388502] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 474.648469] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 474.965810] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 482.282533] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 482.300855] Lustre: Skipped 1 previous similar message [ 482.308877] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 482.310799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 484.072420] Lustre: server umount lustre-OST0000 complete [ 492.515553] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 492.517815] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 492.548683] Lustre: Skipped 2 previous similar messages [ 492.562856] Lustre: Skipped 1 previous similar message [ 495.234937] Lustre: server umount lustre-MDT0000 complete [ 497.637675] LustreError: 15064:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 497.676148] LustreError: 15064:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 500.523230] LustreError: 15853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136098 with bad export cookie 10341427104508515332 [ 500.531791] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 500.541865] LustreError: 15853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 501.049830] Lustre: server umount lustre-MDT0001 complete [ 507.765071] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 10:15:04 (1789136104) [ 516.533063] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 517.216695] LustreError: 19293:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 517.336647] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 522.722551] LustreError: 19294:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.245808] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 527.848068] LustreError: 19293:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 534.372028] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 541.186402] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 551.036645] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 559.205233] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 567.173579] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 577.186612] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 589.536973] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 595.599057] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 606.692755] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 606.703541] LustreError: Skipped 1 previous similar message [ 606.711473] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 606.737411] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 606.747620] Lustre: Skipped 1 previous similar message [ 611.026899] Lustre: server umount lustre-MDT0000 complete [ 613.858513] LustreError: 20022:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 613.882671] LustreError: 20022:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 615.352226] LustreError: 21137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136213 with bad export cookie 10341427104508516207 [ 615.354346] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 615.377351] LustreError: 21137:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 615.948212] Lustre: server umount lustre-MDT0001 complete [ 631.894301] Lustre: server umount lustre-OST0000 complete [ 660.359665] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 672.232910] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 672.801172] LustreError: 23701:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 672.878161] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 672.884453] Lustre: Skipped 2 previous similar messages [ 677.489294] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 688.090790] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 694.394665] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 698.405225] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 705.984486] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 713.969127] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 724.806922] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 724.992767] Lustre: lustre-OST0001: new disk, initializing [ 724.996685] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 725.003667] Lustre: 26017:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 726.962722] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 726.979209] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 727.222096] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 732.753498] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 744.611720] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 749.509716] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 765.417149] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 765.431962] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 765.463108] Lustre: Skipped 2 previous similar messages [ 765.482238] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 765.494394] Lustre: Skipped 3 previous similar messages [ 770.081238] Lustre: server umount lustre-MDT0000 complete [ 773.091703] LustreError: 24634:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 773.119629] LustreError: 24634:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 774.828849] LustreError: 23681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136372 with bad export cookie 10341427104508517061 [ 774.837184] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 774.857963] LustreError: 23681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 775.378795] Lustre: server umount lustre-MDT0001 complete [ 790.578094] Lustre: server umount lustre-OST0000 complete [ 805.081319] Lustre: server umount lustre-OST0001 complete [ 823.042608] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 834.504027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 835.006664] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 835.011137] Lustre: Skipped 3 previous similar messages [ 840.042305] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 840.166449] LustreError: 29037:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 840.187629] LustreError: 29037:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 849.678887] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 855.022480] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 858.294740] Lustre: 30185:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 866.503032] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 876.149852] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 888.266171] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 897.373864] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 906.016730] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 910.317992] Lustre: 32066:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 919.530100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 919.532800] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 919.546488] Lustre: Skipped 6 previous similar messages [ 919.554665] Lustre: Skipped 6 previous similar messages [ 923.908572] Lustre: server umount lustre-MDT0000 complete [ 927.949724] LustreError: 32068:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136525 with bad export cookie 10341427104508518349 [ 927.959298] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 928.342373] Lustre: server umount lustre-MDT0001 complete [ 942.732301] Lustre: server umount lustre-OST0000 complete [ 945.633416] Lustre: 3625:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789136527/real 1789136527] req@ffff95f803e97480 x1876044706800256/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789136543 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 946.987741] Lustre: server umount lustre-OST0001 complete [ 957.998605] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 10:22:34 (1789136554) [ 987.961881] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 998.229720] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 998.709505] LustreError: 34872:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 998.725306] LustreError: 34872:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 11 previous similar messages [ 1002.920442] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1012.714443] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1018.042073] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1021.179806] Lustre: 36021:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1028.895046] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1037.075616] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1046.246409] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1053.212978] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1061.818629] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1066.203774] Lustre: 37902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1074.657473] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1074.668580] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1074.685454] Lustre: Skipped 1 previous similar message [ 1074.691740] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1074.696767] Lustre: Skipped 1 previous similar message [ 1078.796486] Lustre: server umount lustre-MDT0000 complete [ 1083.000237] LustreError: 34853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136680 with bad export cookie 10341427104508519455 [ 1083.016123] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1083.020914] LustreError: 34853:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1083.511762] Lustre: server umount lustre-MDT0001 complete [ 1098.542865] Lustre: server umount lustre-OST0000 complete [ 1114.192905] Lustre: server umount lustre-OST0001 complete [ 1124.623609] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 10:25:21 (1789136721) [ 1142.730156] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1152.642846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1153.284334] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1153.292548] Lustre: Skipped 7 previous similar messages [ 1157.851069] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1166.850392] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1171.373968] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1174.724569] Lustre: 41194:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1181.501444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1189.172800] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1200.047993] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1207.499158] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1215.894536] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1220.234628] Lustre: 43071:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1221.271634] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1221.274608] Lustre: Skipped 1 previous similar message [ 1226.221221] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1226.244307] Lustre: Skipped 5 previous similar messages [ 1230.522927] Lustre: server umount lustre-MDT0000 complete [ 1234.672000] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136832 with bad export cookie 10341427104508520568 [ 1234.684277] LustreError: 40027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1234.684488] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1241.575165] Lustre: server umount lustre-MDT0001 complete [ 1253.507672] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1260.010427] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x8f8421290ad3acdf [ 1260.035626] Lustre: MGC192.168.201.146@tcp: Connection restored to 0@lo (at 0@lo) [ 1260.560150] LustreError: 41557:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1260.576195] LustreError: 41557:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 28 previous similar messages [ 1264.793393] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1269.936925] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1274.427731] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1279.802167] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1279.981060] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1280.007086] Lustre: Skipped 1 previous similar message [ 1287.843742] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1289.753451] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1298.010768] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1300.798832] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1310.728523] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1314.827937] LustreError: 45693:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (ffff95f91fd16400) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 1315.179202] Lustre: server umount lustre-MDT0000 complete [ 1319.978563] LustreError: 43280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789136917 with bad export cookie 10341427104508521695 [ 1319.989076] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1319.996121] LustreError: 43280:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1320.545049] Lustre: server umount lustre-MDT0001 complete [ 1335.616171] Lustre: server umount lustre-OST0000 complete [ 1336.096982] Lustre: 3622:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789136918/real 1789136918] req@ffff95f905d7fc00 x1876044706942208/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789136934 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1339.469364] Lustre: server umount lustre-OST0001 complete [ 1349.893938] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 10:29:06 (1789136946) [ 1367.166058] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1377.502206] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1382.746608] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1392.233650] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1397.927512] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1401.164889] Lustre: 48835:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1407.985936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1416.775987] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1425.882521] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1432.785876] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1440.319974] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1445.302051] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1460.194741] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1460.204354] Lustre: Skipped 10 previous similar messages [ 1460.205372] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1460.213762] Lustre: Skipped 18 previous similar messages [ 1466.136489] Lustre: server umount lustre-MDT0000 complete [ 1470.628113] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137068 with bad export cookie 10341427104508522493 [ 1470.638178] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1470.648308] LustreError: 47667:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1471.215650] Lustre: server umount lustre-MDT0001 complete [ 1486.154437] Lustre: server umount lustre-OST0000 complete [ 1499.552289] Lustre: server umount lustre-OST0001 complete [ 1510.731230] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 10:31:47 (1789137107) [ 1513.120798] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1515.341037] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 10:31:52 (1789137112) [ 1527.249124] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 1536.646770] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1559.101979] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1559.311034] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1559.384290] Lustre: lustre-MDT0000: new disk, initializing [ 1559.463076] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1563.851621] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1574.652820] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1583.057220] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1583.317516] Lustre: lustre-OST0000: new disk, initializing [ 1583.320863] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1583.326933] Lustre: Skipped 1 previous similar message [ 1583.330943] Lustre: 54912:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1585.054441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1585.069109] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1585.138847] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1590.072830] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1601.816657] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1604.884749] Lustre: DEBUG MARKER: create test files [ 1641.441707] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1647.004869] Lustre: server umount lustre-OST0000 complete [ 1651.411184] Lustre: server umount lustre-MDT0000 complete [ 1664.130636] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 1667.054844] Key type lgssc unregistered [ 1667.409982] LNet: 56551:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1667.414844] LNetError: 56551:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1667.455984] LNet: Removed LNI 192.168.201.146@tcp [ 1668.382166] Key type .llcrypt unregistered [ 1668.384062] Key type ._llcrypt unregistered [ 1698.881366] Key type ._llcrypt registered [ 1698.883348] Key type .llcrypt registered [ 1698.962714] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1700.364243] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1700.376300] alg: No test for adler32 (adler32-zlib) [ 1701.409330] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 1701.669352] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 1703.360391] Key type lgssc registered [ 1704.423050] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1711.877404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1711.899838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1713.398987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1717.924616] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1726.782312] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1735.517771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1736.032922] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1738.105927] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:101 to 0x240000400:129) [ 1738.257624] Lustre: 59289:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1738.270580] Lustre: 59289:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1738.277286] Lustre: 59289:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1738.287923] Lustre: 59289:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1738.297375] Lustre: 59289:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1738.313030] Lustre: 59289:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1744.201944] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1753.211979] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1759.390366] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1766.271274] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1799.654618] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1799.667390] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1799.681550] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1804.364151] Lustre: server umount lustre-OST0000 complete [ 1808.440982] Lustre: server umount lustre-MDT0000 complete [ 1819.529685] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 1822.357574] Key type lgssc unregistered [ 1822.623705] LNet: 60929:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1822.634436] LNetError: 60929:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1822.650386] LNet: Removed LNI 192.168.201.146@tcp [ 1823.533205] Key type .llcrypt unregistered [ 1823.536149] Key type ._llcrypt unregistered [ 1854.009026] Key type ._llcrypt registered [ 1854.017499] Key type .llcrypt registered [ 1854.102849] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1855.284131] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1855.323187] alg: No test for adler32 (adler32-zlib) [ 1856.402435] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 1856.560403] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 1858.200177] Key type lgssc registered [ 1859.408688] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1867.602450] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1867.615258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1869.157194] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1874.025766] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1882.207704] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1889.512170] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1889.997030] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1894.122336] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1894.266552] Lustre: 63769:0:(osd_internal.h:1463:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1894.278488] Lustre: 63769:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1894.294571] Lustre: 63769:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1894.305982] Lustre: 63769:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1894.325280] Lustre: 63769:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1894.337447] Lustre: 63769:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1898.554792] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 1907.421901] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1913.433190] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1925.089693] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1925.115459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1928.520922] Lustre: server umount lustre-OST0000 complete [ 1932.385364] Lustre: server umount lustre-MDT0000 complete [ 1944.166464] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 1947.368763] Key type lgssc unregistered [ 1947.687717] LNet: 65332:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1947.700174] LNetError: 65332:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1948.710501] LNet: Removed LNI 192.168.201.146@tcp [ 1949.612977] Key type .llcrypt unregistered [ 1949.621689] Key type ._llcrypt unregistered [ 1971.402307] Key type ._llcrypt registered [ 1971.410963] Key type .llcrypt registered [ 1971.578566] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 1989.707236] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 1992.318381] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1992.359306] alg: No test for adler32 (adler32-zlib) [ 1993.542674] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 1993.889331] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 1995.652133] Key type lgssc registered [ 1996.974320] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2044.027466] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2044.059738] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2045.332692] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2045.350533] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2045.404877] Lustre: lustre-MDT0000: new disk, initializing [ 2045.529272] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2045.553793] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2050.442320] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2061.815711] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2061.934579] Lustre: 68487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2061.985576] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2061.990672] Lustre: Skipped 1 previous similar message [ 2062.086985] Lustre: lustre-MDT0001: new disk, initializing [ 2062.185738] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2062.217397] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2062.233595] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2067.233871] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2077.761805] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2084.282635] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2092.869370] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2093.215211] Lustre: lustre-OST0000: new disk, initializing [ 2093.218254] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2093.226463] Lustre: 70459:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2093.345476] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2100.287271] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2102.912318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2102.940082] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2103.124451] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2113.498490] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2120.082365] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2120.414555] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2126.693602] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2127.021439] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2133.987659] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2133.992879] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2134.000911] Lustre: Skipped 1 previous similar message [ 2134.001316] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2134.022972] Lustre: Skipped 1 previous similar message [ 2135.535475] Lustre: server umount lustre-OST0000 complete [ 2144.226581] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2144.227422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2144.265531] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2146.746527] Lustre: server umount lustre-MDT0000 complete [ 2151.011716] LustreError: 68481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137748 with bad export cookie 6503174773855619700 [ 2151.014514] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2151.027770] LustreError: 68481:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2151.473901] Lustre: server umount lustre-MDT0001 complete [ 2160.998437] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 10:42:37 (1789137757) [ 2173.434142] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 2218.425256] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 2230.047437] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 2274.181730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2274.379874] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2274.401802] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2274.461225] Lustre: lustre-MDT0000: new disk, initializing [ 2274.524359] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2274.552921] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2279.845420] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2291.477825] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2291.621491] Lustre: 76487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2291.696778] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2291.700698] Lustre: Skipped 1 previous similar message [ 2291.814810] Lustre: lustre-MDT0001: new disk, initializing [ 2291.931152] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2291.972540] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2291.982691] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2297.062889] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2307.352911] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2313.737793] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2324.349497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2324.686384] Lustre: lustre-OST0000: new disk, initializing [ 2324.692663] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2324.705439] Lustre: 78457:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2324.837279] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2330.258806] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2330.267410] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2330.430528] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2333.794587] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2346.209580] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2352.151678] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2352.515752] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2358.844413] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2359.212637] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2366.433358] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2366.443168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2366.454026] LustreError: Skipped 1 previous similar message [ 2366.461800] Lustre: Skipped 1 previous similar message [ 2366.477624] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2366.496686] Lustre: Skipped 2 previous similar messages [ 2368.207260] Lustre: server umount lustre-OST0000 complete [ 2374.113791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2374.121603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2374.132420] Lustre: Skipped 1 previous similar message [ 2374.140122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2379.163664] Lustre: server umount lustre-MDT0000 complete [ 2381.796628] LustreError: 78050:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2381.808025] LustreError: 78050:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2383.666895] LustreError: 77292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789137981 with bad export cookie 6503174773855620589 [ 2383.674236] LustreError: 77292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2383.676816] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2384.166951] Lustre: server umount lustre-MDT0001 complete [ 2396.045941] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 10:46:32 (1789137992) [ 2405.592776] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2406.377371] LustreError: 80873:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2406.487873] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2411.492912] LustreError: 80874:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2411.741735] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2416.613648] LustreError: 80873:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2421.732333] LustreError: 80874:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2423.892763] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2431.469981] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2441.806755] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2448.129828] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2457.035248] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2457.677649] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2457.684047] Lustre: Skipped 1 previous similar message [ 2465.176473] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2475.045684] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2480.598575] Lustre: 83548:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting 3c6bceb5-44f1-4798-94fd-8f3e5092d034 at adminstrative request [ 2481.142118] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2481.162045] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2491.362961] Lustre: MGS: Client 3c6bceb5-44f1-4798-94fd-8f3e5092d034 (at 0@lo) reconnecting [ 2491.370777] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x5a3feb0063f2115f to 0x5a3feb0063f213ff [ 2491.398098] Lustre: MGC192.168.201.146@tcp: Connection restored to 0@lo (at 0@lo) [ 2520.292688] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2520.524364] Lustre: lustre-OST0001: new disk, initializing [ 2520.533309] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2520.541035] Lustre: 83940:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2527.870571] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2527.889614] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2528.122951] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2528.211784] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2541.206425] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2548.706612] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2548.706831] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2548.717245] LustreError: Skipped 1 previous similar message [ 2548.734158] Lustre: Skipped 1 previous similar message [ 2548.735113] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2548.735118] Lustre: Skipped 2 previous similar messages [ 2551.032371] Lustre: server umount lustre-OST0001 complete [ 2553.830764] LustreError: 82696:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2553.863518] LustreError: 82696:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 2556.900672] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2556.908796] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2556.919313] Lustre: Skipped 1 previous similar message [ 2556.938118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2556.951031] Lustre: Skipped 1 previous similar message [ 2561.879582] Lustre: server umount lustre-OST0000 complete [ 2572.567443] Lustre: server umount lustre-MDT0000 complete [ 2574.306696] LustreError: 80869:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2574.341965] LustreError: 80869:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 2577.139276] LustreError: 81608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138175 with bad export cookie 6503174773855622143 [ 2577.149361] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2577.153660] LustreError: 81608:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2577.462333] Lustre: server umount lustre-MDT0001 complete [ 2587.919688] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 10:49:44 (1789138184) [ 2589.881637] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2591.777673] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 10:49:48 (1789138188) [ 2593.495472] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2595.346433] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 10:49:52 (1789138192) [ 2597.110431] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2599.451029] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 10:49:55 (1789138195) [ 2601.330792] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2603.548230] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 10:50:00 (1789138200) [ 2605.452025] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2612.250677] loop: module loaded [ 2612.263729] loop0: detected capacity change from 0 to 4294967296 [ 2612.368428] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2613.800855] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2617.974788] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2622.526819] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2649.542585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2649.609124] LustreError: 88063:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2649.615251] LustreError: 88063:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2649.619931] LustreError: 88063:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2649.622640] LustreError: 88063:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2674.670950] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 2686.031880] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 2730.055137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2730.394209] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2730.420619] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2730.515318] Lustre: lustre-MDT0000: new disk, initializing [ 2730.602542] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2730.609090] Lustre: Skipped 1 previous similar message [ 2730.625926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2736.858476] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2750.025719] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2750.157407] Lustre: 90932:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2750.545198] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2757.073294] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2769.562497] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2778.648944] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2787.494866] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2787.940419] Lustre: 92905:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2795.322398] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2795.609288] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2795.624766] Lustre: Skipped 1 previous similar message [ 2795.629824] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2795.723887] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2806.207364] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2813.500025] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2813.866588] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2820.461977] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2820.899350] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2826.726212] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2826.726470] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2826.730292] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2826.730302] Lustre: Skipped 6 previous similar messages [ 2826.754426] LustreError: Skipped 3 previous similar messages [ 2826.776272] Lustre: Skipped 4 previous similar messages [ 2829.636493] Lustre: server umount lustre-OST0000 complete [ 2840.722144] Lustre: server umount lustre-MDT0000 complete [ 2842.086777] LustreError: 92084:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2845.599545] LustreError: 90923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138443 with bad export cookie 6503174773855622759 [ 2845.617799] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2846.466873] Lustre: server umount lustre-MDT0001 complete [ 2856.092933] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 10:54:12 (1789138452) [ 2863.524067] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2864.157593] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 2864.162129] Lustre: Skipped 2 previous similar messages [ 2869.030689] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2879.834585] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2885.043893] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2893.316954] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2899.719062] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2907.217720] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2914.200741] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2922.718671] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2928.170747] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2928.461042] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2933.964753] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2934.262806] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2941.474671] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2941.747069] Lustre: lustre-OST0001: new disk, initializing [ 2941.752219] Lustre: Skipped 2 previous similar messages [ 2941.765108] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2941.770407] Lustre: Skipped 3 previous similar messages [ 2941.776701] Lustre: 98689:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2949.113348] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 2950.811563] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2950.821637] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2950.901596] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2960.231573] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2966.694946] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2966.926937] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2971.774311] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2972.047155] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2993.917299] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 3019.828292] Lustre: Failing over lustre-MDT0000 [ 3020.238398] Lustre: server umount lustre-MDT0000 complete [ 3021.988952] LustreError: 95311:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 192.168.201.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3022.018398] LustreError: 95311:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 3022.817903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3022.834452] Lustre: Skipped 2 previous similar messages [ 3039.200156] Lustre: 66710:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789138620/real 1789138620] req@00000000290e07b3 x1876046738913664/t0(0) o400->MGC192.168.201.146@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1789138636 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 3039.218686] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3045.643342] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 3045.646194] LDISKFS-fs (dm-0): recovery complete [ 3045.655120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3049.446276] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x5a3feb0063f2de7a [ 3049.469698] Lustre: MGC192.168.201.146@tcp: Connection restored to 0@lo (at 0@lo) [ 3049.882121] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 3053.690586] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 3053.690655] Lustre: 95314:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3055.099087] Lustre: 95317:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 3055.118931] Lustre: 95317:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 1 previous similar message [ 3055.136377] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3055.265202] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3055.337808] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3055.608128] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3055.949760] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3055.958877] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3056.284856] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3056.295602] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 1 previous similar message [ 3056.301168] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3056.318983] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 7 previous similar messages [ 3056.981398] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3056.990831] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 3057.313110] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3057.319496] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 2 previous similar messages [ 3058.305294] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3058.323130] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 5 previous similar messages [ 3059.321896] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3059.340245] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 3059.672877] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3059.687281] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 6 previous similar messages [ 3062.560124] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3062.567956] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 12 previous similar messages [ 3063.548493] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3063.557898] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3063.888240] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3063.900497] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 12 previous similar messages [ 3070.561820] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3070.571100] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 24 previous similar messages [ 3071.857511] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3071.865886] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 3072.184181] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3072.193578] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 25 previous similar messages [ 3086.620919] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3086.635819] Lustre: 101538:0:(ldlm_lib.c:2148:extend_recovery_timer()) Skipped 49 previous similar messages [ 3087.939449] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3087.955919] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 49 previous similar messages [ 3088.280162] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3088.301165] LustreError: 101538:0:(ldlm_lib.c:2761:replay_request_or_update()) Skipped 49 previous similar messages [ 3113.224841] Lustre: 101538:0:(ldlm_lib.c:2457:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3113.231115] Lustre: 101538:0:(ldlm_lib.c:2467:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3113.238111] Lustre: 101538:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 4a5cdf3f-2d27-43db-9982-401577175acf@192.168.201.46@tcp [ 3113.251135] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3113.279605] Lustre: 101538:0:(ldlm_lib.c:1977:abort_req_replay_queue()) @@@ aborted: req@00000000f525d95e x1876046718769152/t0(8589934777) o101->4a5cdf3f-2d27-43db-9982-401577175acf@192.168.201.46@tcp:121/0 lens 592/0 e 0 to 0 dl 1789138721 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3113.316183] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3113.331336] Lustre: lustre-MDT0000: Denying connection for new client 4a5cdf3f-2d27-43db-9982-401577175acf (at 192.168.201.46@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3113.356817] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3113.420909] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3113.423432] Lustre: Skipped 2 previous similar messages [ 3113.432583] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3113.467881] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:47 to 0x2c0000401:545) [ 3113.474769] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:79 to 0x280000401:577) [ 3130.852635] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3130.862171] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3130.872729] Lustre: Skipped 3 previous similar messages [ 3130.895288] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3130.910458] Lustre: Skipped 3 previous similar messages [ 3136.227286] Lustre: server umount lustre-OST0000 complete [ 3152.357455] LustreError: 98401:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3152.388619] LustreError: 98401:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 47 previous similar messages [ 3154.401464] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3154.696465] Lustre: server umount lustre-OST0001 complete [ 3165.990922] Lustre: server umount lustre-MDT0000 complete [ 3171.065645] LustreError: 95299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789138769 with bad export cookie 6503174773855673978 [ 3171.069250] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3171.080084] LustreError: 95299:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3171.579959] Lustre: server umount lustre-MDT0001 complete [ 3182.154793] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 10:59:38 (1789138778) [ 3190.832240] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3190.838885] Lustre: *** cfs_fail_loc=197, val=0*** [ 3206.432802] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3211.552685] LustreError: 103516:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.146@tcp: failed processing log, type 4: rc = -110 [ 3216.686638] Lustre: *** cfs_fail_loc=197, val=0*** [ 3216.691174] Lustre: Skipped 65 previous similar messages [ 3237.280883] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3237.293368] Lustre: Skipped 4 previous similar messages [ 3244.743553] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3254.064410] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3256.397128] Lustre: server umount lustre-OST0000 complete [ 3267.508937] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 11:01:03 (1789138863) [ 3289.753434] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3294.653332] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3303.496870] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3307.903798] LustreError: 105179:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3307.932763] LustreError: 105179:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3307.946860] LustreError: 105179:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 17, retries 0, failed: rc = -5 [ 3308.582803] Lustre: server umount lustre-MDT0000 complete [ 3354.931835] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 3370.607353] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 3416.823077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3417.336183] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3417.421970] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3417.559300] Lustre: lustre-MDT0000: new disk, initializing [ 3417.711743] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3424.478211] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3438.565419] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3438.780690] Lustre: 109378:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3439.063557] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3443.996000] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3456.287866] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3464.307877] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3472.548176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3472.956568] Lustre: 111348:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3478.291426] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3482.066285] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3494.182581] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3501.317065] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3501.687263] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3507.562422] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3507.801057] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3514.340158] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3514.346142] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3514.354754] Lustre: Skipped 5 previous similar messages [ 3514.365958] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3514.371249] LustreError: Skipped 3 previous similar messages [ 3514.380544] Lustre: Skipped 14 previous similar messages [ 3516.879114] Lustre: server umount lustre-OST0000 complete [ 3536.352244] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3536.650360] Lustre: server umount lustre-MDT0000 complete [ 3539.940218] LustreError: 109388:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3539.948662] LustreError: 109388:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [ 3540.858687] LustreError: 110182:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139138 with bad export cookie 6503174773855684632 [ 3540.873871] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3541.317389] Lustre: server umount lustre-MDT0001 complete [ 3555.111957] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 11:05:50 (1789139150) [ 3569.682582] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 3572.954780] Key type lgssc unregistered [ 3573.328536] LNet: 113844:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3573.341846] LNetError: 113844:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3573.377300] LNet: Removed LNI 192.168.201.146@tcp [ 3574.478882] Key type .llcrypt unregistered [ 3574.481167] Key type ._llcrypt unregistered [ 3594.356860] Key type ._llcrypt registered [ 3594.359353] Key type .llcrypt registered [ 3594.441874] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 3613.885843] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 3615.177042] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3615.320762] alg: No test for adler32 (adler32-zlib) [ 3616.426481] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 3616.712797] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 3618.424205] Key type lgssc registered [ 3619.913597] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3668.629627] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3668.689656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3670.099197] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3670.143769] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3670.263364] Lustre: lustre-MDT0000: new disk, initializing [ 3670.406575] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3670.425937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3675.539176] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3688.969461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3689.533686] Lustre: lustre-OST0000: new disk, initializing [ 3689.542857] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3689.553779] Lustre: Skipped 1 previous similar message [ 3689.560489] Lustre: 118022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3689.746661] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3697.219921] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3697.789194] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3697.796370] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3697.840288] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3710.587440] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3710.866741] Lustre: lustre-OST0001: new disk, initializing [ 3710.872183] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3710.879776] Lustre: 118885:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3710.987284] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3717.329411] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3717.351559] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3717.403141] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3720.429529] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3732.964594] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3732.970529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3732.991619] Lustre: Skipped 1 previous similar message [ 3733.009658] Lustre: Skipped 1 previous similar message [ 3737.484988] Lustre: server umount lustre-MDT0000 complete [ 3744.983216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3750.199846] LustreError: 117208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139348 with bad export cookie 12635205575590682640 [ 3750.205168] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3750.213672] LustreError: 117208:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3750.326249] Lustre: server umount lustre-OST0000 complete [ 3755.366739] Lustre: server umount lustre-OST0001 complete [ 3765.117380] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 11:09:21 (1789139361) [ 3796.279328] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 11:09:52 (1789139392) [ 3811.192756] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 3844.579454] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 11:10:40 (1789139440) [ 3898.081341] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 3918.803244] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 3962.801468] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3963.273456] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3963.314690] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3963.463512] Lustre: lustre-MDT0000: new disk, initializing [ 3963.555420] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3963.583158] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3970.278329] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 3984.607743] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3984.858163] Lustre: 126683:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3985.031046] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3985.049154] Lustre: Skipped 1 previous similar message [ 3985.236049] Lustre: lustre-MDT0001: new disk, initializing [ 3985.426206] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3985.497641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3985.524023] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3990.971751] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4002.556085] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4009.768989] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4018.765197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4019.139507] Lustre: lustre-OST0000: new disk, initializing [ 4019.147790] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4019.158866] Lustre: 128653:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4019.286249] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4027.095983] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4027.993757] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4028.012600] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4028.193069] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4039.455993] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4046.259498] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 4046.493440] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 4052.213639] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 4052.452892] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 4059.106466] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4059.111806] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4059.131400] Lustre: Skipped 1 previous similar message [ 4059.148731] Lustre: Skipped 1 previous similar message [ 4064.227386] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4064.241036] Lustre: Skipped 1 previous similar message [ 4069.344154] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4069.352455] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4069.369830] Lustre: Skipped 1 previous similar message [ 4069.777273] Lustre: server umount lustre-OST0000 complete [ 4074.465826] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4074.495978] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4074.497026] Lustre: Skipped 1 previous similar message [ 4074.513468] Lustre: Skipped 1 previous similar message [ 4080.810323] Lustre: server umount lustre-MDT0000 complete [ 4085.259299] LustreError: 128652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139683 with bad export cookie 12635205575590683865 [ 4085.286155] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4085.287127] LustreError: 128652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4086.049701] Lustre: server umount lustre-MDT0001 complete [ 4098.145792] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 11:14:54 (1789139694) [ 4107.357391] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4108.117364] LustreError: 131066:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4108.139987] LustreError: 131066:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 4108.228669] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4113.394045] LustreError: 131067:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4113.953087] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4118.504570] LustreError: 131066:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4123.627862] LustreError: 131067:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4125.590213] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4133.338861] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4145.820362] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4153.579869] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4163.571873] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4163.778855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4163.781639] Lustre: Skipped 1 previous similar message [ 4171.507565] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4180.953326] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4186.345070] Lustre: *** cfs_fail_loc=159, val=0*** [ 4202.687765] Lustre: lustre-MDT0000: Client 1d97fad9-b66a-43b6-b0d6-32e642d8a04f (at 192.168.201.46@tcp) reconnecting [ 4204.922306] Lustre: *** cfs_fail_loc=159, val=0*** [ 4204.925432] Lustre: Skipped 5 previous similar messages [ 4221.146126] Lustre: lustre-MDT0000: Client 1d97fad9-b66a-43b6-b0d6-32e642d8a04f (at 192.168.201.46@tcp) reconnecting [ 4239.843246] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4239.845324] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4239.848313] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4239.848321] Lustre: Skipped 2 previous similar messages [ 4239.873606] Lustre: Skipped 1 previous similar message [ 4253.152170] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4253.350636] Lustre: server umount lustre-OST0000 complete [ 4259.297789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4259.306603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4259.337937] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4259.361368] Lustre: Skipped 5 previous similar messages [ 4265.446366] LustreError: 131066:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4265.475860] LustreError: 131066:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4265.499290] Lustre: server umount lustre-MDT0000 complete [ 4271.314919] LustreError: 131047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789139869 with bad export cookie 12635205575590684747 [ 4271.343760] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4271.344988] LustreError: 131047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4272.031928] Lustre: server umount lustre-MDT0001 complete [ 4290.453349] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 4294.452716] Key type lgssc unregistered [ 4294.925668] LNet: 134876:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4294.942752] LNetError: 134876:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4294.960305] LNet: Removed LNI 192.168.201.146@tcp [ 4296.129405] Key type .llcrypt unregistered [ 4296.135876] Key type ._llcrypt unregistered [ 4332.953970] Key type ._llcrypt registered [ 4332.956898] Key type .llcrypt registered [ 4333.075560] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 4334.684774] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4334.711568] alg: No test for adler32 (adler32-zlib) [ 4335.923345] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 4336.322732] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 4338.185138] Key type lgssc registered [ 4340.052781] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4351.345510] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4351.370173] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4353.152961] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4359.219080] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4363.751049] LustreError: 136293:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4363.774354] LustreError: 136293:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4368.877560] LustreError: 136294:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4369.415718] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4369.826588] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4376.186863] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4386.091247] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4393.908653] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4401.593594] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4402.058620] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4411.148921] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4420.975311] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4423.609668] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4437.706971] Lustre: *** cfs_fail_loc=159, val=0*** [ 4454.078607] Lustre: lustre-MDT0000: Client 3e176e0b-04a7-42db-ab93-a4a8f0450bfe (at 192.168.201.46@tcp) reconnecting [ 4458.072547] Lustre: *** cfs_fail_loc=159, val=0*** [ 4474.572775] Lustre: lustre-MDT0001: Client 3e176e0b-04a7-42db-ab93-a4a8f0450bfe (at 192.168.201.46@tcp) reconnecting [ 4476.617979] Lustre: *** cfs_fail_loc=159, val=0*** [ 4476.635377] Lustre: Skipped 3 previous similar messages [ 4491.982782] Lustre: lustre-MDT0001: Client 3e176e0b-04a7-42db-ab93-a4a8f0450bfe (at 192.168.201.46@tcp) reconnecting [ 4508.767133] Lustre: 139382:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4511.133411] Lustre: *** cfs_fail_loc=159, val=0*** [ 4526.836084] Lustre: lustre-MDT0000: Client f3499b56-7d6f-45d7-a703-728dcc8f1de5 (at 192.168.201.46@tcp) reconnecting [ 4528.894509] Lustre: *** cfs_fail_loc=159, val=0*** [ 4528.903565] Lustre: Skipped 14 previous similar messages [ 4545.219288] Lustre: lustre-MDT0000: Client f3499b56-7d6f-45d7-a703-728dcc8f1de5 (at 192.168.201.46@tcp) reconnecting [ 4546.884990] Lustre: 139670:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4556.771834] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4556.781359] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4556.812410] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4556.829734] Lustre: Skipped 1 previous similar message [ 4560.150310] Lustre: server umount lustre-OST0000 complete [ 4567.010063] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4567.016161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4567.023478] Lustre: Skipped 2 previous similar messages [ 4567.038028] Lustre: Skipped 1 previous similar message [ 4571.163403] Lustre: server umount lustre-MDT0000 complete [ 4572.130821] LustreError: 136294:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4572.155343] LustreError: 136294:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4576.092621] LustreError: 139261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140174 with bad export cookie 9626203025796932956 [ 4576.096152] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4576.111097] LustreError: 139261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4576.852910] Lustre: server umount lustre-MDT0001 complete [ 4589.745246] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 4592.901551] Key type lgssc unregistered [ 4593.271668] LNet: 140729:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4593.275865] LNetError: 140729:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4593.290852] LNet: Removed LNI 192.168.201.146@tcp [ 4594.289670] Key type .llcrypt unregistered [ 4594.294303] Key type ._llcrypt unregistered [ 4622.731375] Key type ._llcrypt registered [ 4622.734909] Key type .llcrypt registered [ 4622.961192] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 4624.452729] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4624.619109] alg: No test for adler32 (adler32-zlib) [ 4625.617588] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 4625.917577] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 4627.640209] Key type lgssc registered [ 4628.759686] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4637.926375] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4637.939977] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4639.385202] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4643.814629] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4649.959049] LustreError: 142144:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4649.983080] LustreError: 142144:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4653.113282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4653.569742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4659.237098] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4667.890522] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4674.361357] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4682.002954] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4682.381162] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4684.503188] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4689.303341] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4697.537835] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4699.826207] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4705.007631] Lustre: 144728:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4708.986809] Lustre: 144777:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4710.546089] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4711.428515] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4715.394730] Lustre: 144923:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4716.369220] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4717.282907] Lustre: 145021:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4725.731322] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4725.737465] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4725.738508] LustreError: Skipped 1 previous similar message [ 4725.756309] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4725.776751] Lustre: Skipped 1 previous similar message [ 4728.417848] Lustre: server umount lustre-OST0000 complete [ 4735.466707] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4735.486523] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4735.518907] Lustre: Skipped 1 previous similar message [ 4735.536790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4740.059747] Lustre: server umount lustre-MDT0000 complete [ 4741.094413] LustreError: 142145:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4741.131619] LustreError: 142145:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4745.203987] LustreError: 142140:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4745.231937] LustreError: 142140:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4746.064618] LustreError: 142124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140344 with bad export cookie 4615006369581194623 [ 4746.071927] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4746.081381] LustreError: 142124:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4746.701774] Lustre: server umount lustre-MDT0001 complete [ 4760.373942] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 4763.526693] Key type lgssc unregistered [ 4763.873414] LNet: 146087:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4763.877733] LNetError: 146087:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4763.891167] LNet: Removed LNI 192.168.201.146@tcp [ 4764.925501] Key type .llcrypt unregistered [ 4764.926958] Key type ._llcrypt unregistered [ 4794.279603] Key type ._llcrypt registered [ 4794.284804] Key type .llcrypt registered [ 4794.388765] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 4796.016175] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4796.193404] alg: No test for adler32 (adler32-zlib) [ 4797.443151] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 4797.770583] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 4799.448222] Key type lgssc registered [ 4801.188195] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4811.206378] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4811.214593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4812.792153] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4819.585733] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4823.016046] LustreError: 147502:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4823.053470] LustreError: 147502:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 4828.131424] LustreError: 147503:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4830.526692] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4830.991122] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4836.029206] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4845.105467] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4851.925671] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4861.937113] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4862.788583] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4872.976062] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 4876.132134] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4882.298509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4885.308651] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4892.470799] Lustre: *** cfs_fail_loc=159, val=0*** [ 4892.482265] Lustre: Skipped 1 previous similar message [ 4907.734492] Lustre: lustre-MDT0000: Client 1eedae14-467a-404c-9591-40928dda9d9c (at 192.168.201.46@tcp) reconnecting [ 4927.460073] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4927.485303] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4927.514096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4928.486073] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4928.489612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4928.495807] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4928.526363] Lustre: Skipped 1 previous similar message [ 4931.522215] Lustre: server umount lustre-OST0000 complete [ 4938.721598] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4938.776841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4938.789919] Lustre: Skipped 1 previous similar message [ 4942.106061] Lustre: server umount lustre-MDT0000 complete [ 4943.843574] LustreError: 148799:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4943.855995] LustreError: 148799:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 4945.968821] LustreError: 147484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140543 with bad export cookie 17874190607963999757 [ 4945.976364] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4945.982658] LustreError: 147484:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4946.523332] Lustre: server umount lustre-MDT0001 complete [ 4959.657272] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 4963.022422] Key type lgssc unregistered [ 4963.350770] LNet: 151186:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4963.368673] LNetError: 151186:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4963.401670] LNet: Removed LNI 192.168.201.146@tcp [ 4964.590075] Key type .llcrypt unregistered [ 4964.594727] Key type ._llcrypt unregistered [ 4995.124215] Key type ._llcrypt registered [ 4995.129469] Key type .llcrypt registered [ 4995.251630] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 4996.826774] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4996.851310] alg: No test for adler32 (adler32-zlib) [ 4997.886431] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 4998.145892] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 4999.832249] Key type lgssc registered [ 5000.967296] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5009.619303] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5009.693452] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5011.433536] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5017.341740] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5021.670568] LustreError: 152603:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5021.693665] LustreError: 152603:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5026.786088] LustreError: 152604:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5028.348781] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5029.076133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5035.027574] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5046.878622] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5055.509554] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5066.198695] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5066.916109] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5073.084127] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 5074.923764] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5081.284574] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 5084.289540] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5104.786344] Lustre: lustre-MDT0001: evicting 315b11d2-3888-427a-b1c8-442e88c2a20b (at 192.168.201.46@tcp) by administrative request [ 5104.801430] Lustre: Skipped 1 previous similar message [ 5134.818115] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5134.826518] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5134.848735] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5137.890418] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5137.908613] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5137.924091] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5140.235289] Lustre: server umount lustre-OST0000 complete [ 5146.595925] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5146.607089] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5146.618459] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5146.622895] Lustre: Skipped 1 previous similar message [ 5150.888818] Lustre: server umount lustre-MDT0000 complete [ 5153.249944] LustreError: 152598:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5153.274226] LustreError: 152598:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5155.428708] LustreError: 152585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140753 with bad export cookie 5485898098571960707 [ 5155.432532] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5155.442230] LustreError: 152585:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5155.841839] Lustre: server umount lustre-MDT0001 complete [ 5172.654410] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 5177.859887] Key type lgssc unregistered [ 5178.213837] LNet: 156681:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5178.217814] LNetError: 156681:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5178.250399] LNet: Removed LNI 192.168.201.146@tcp [ 5179.665544] Key type .llcrypt unregistered [ 5179.667727] Key type ._llcrypt unregistered [ 5273.019309] Key type ._llcrypt registered [ 5273.022280] Key type .llcrypt registered [ 5273.274361] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 5275.195470] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5275.291965] alg: No test for adler32 (adler32-zlib) [ 5276.486543] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 5277.054127] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 5278.929899] Key type lgssc registered [ 5282.453353] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5295.181323] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5295.212911] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5296.723115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5303.473855] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5306.852904] LustreError: 159172:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5306.873487] LustreError: 159172:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5311.988594] LustreError: 159173:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5315.194856] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5315.453306] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5320.725778] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5330.952093] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5338.971249] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5349.087050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5349.902828] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5359.710928] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5360.147252] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5366.278446] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5369.736258] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5380.580693] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5380.587478] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5380.597552] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5381.604287] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5381.611440] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5381.620237] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5382.932874] Lustre: server umount lustre-OST0000 complete [ 5391.841386] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5391.857479] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5391.867019] Lustre: Skipped 2 previous similar messages [ 5394.298290] Lustre: server umount lustre-MDT0000 complete [ 5396.962796] LustreError: 159168:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5396.977109] LustreError: 159168:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 5399.498149] LustreError: 161026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789140997 with bad export cookie 1137971899349842423 [ 5399.502677] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5399.517666] LustreError: 161026:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5400.422331] Lustre: server umount lustre-MDT0001 complete [ 5416.174793] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 5420.426918] Key type lgssc unregistered [ 5420.850061] LNet: 162721:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5420.856209] LNetError: 162721:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5420.878728] LNet: Removed LNI 192.168.201.146@tcp [ 5422.216804] Key type .llcrypt unregistered [ 5422.218898] Key type ._llcrypt unregistered [ 5457.820768] Key type ._llcrypt registered [ 5457.823702] Key type .llcrypt registered [ 5457.932461] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 5459.747788] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5459.776587] alg: No test for adler32 (adler32-zlib) [ 5461.014671] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 5461.404325] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 5463.168166] Key type lgssc registered [ 5465.148983] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5490.417894] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 11:38:06 (1789141086) [ 5511.560710] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 5523.038307] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 5565.391253] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5565.412850] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5566.626649] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5566.656770] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5566.720852] Lustre: lustre-MDT0000: new disk, initializing [ 5566.791205] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5566.815735] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5573.379750] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5587.076535] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5587.266718] Lustre: 166996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5587.329816] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5587.337456] Lustre: Skipped 1 previous similar message [ 5587.490680] Lustre: lustre-MDT0001: new disk, initializing [ 5587.660164] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5587.744990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5587.766661] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5594.472864] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5605.523156] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5613.089627] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5621.833864] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5622.394751] Lustre: lustre-OST0000: new disk, initializing [ 5622.401773] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5622.414646] Lustre: 168964:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5622.567126] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5630.219926] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5630.583336] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5630.598392] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5630.795271] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5642.414637] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5656.547499] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5656.561902] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5656.566553] LustreError: Skipped 1 previous similar message [ 5656.576163] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5656.602131] Lustre: Skipped 1 previous similar message [ 5659.682396] Lustre: server umount lustre-OST0000 complete [ 5666.785586] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5666.833529] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5666.851115] Lustre: Skipped 2 previous similar messages [ 5672.016544] Lustre: server umount lustre-MDT0000 complete [ 5677.096324] LustreError: 167009:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5677.140365] LustreError: 167009:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [ 5678.096747] LustreError: 166989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789141276 with bad export cookie 392185519887524486 [ 5678.098810] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5678.118885] LustreError: 166989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5678.818309] Lustre: server umount lustre-MDT0001 complete [ 5695.148865] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 5698.446201] Key type lgssc unregistered [ 5698.872521] LNet: 170873:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5698.878640] LNetError: 170873:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5698.892458] LNet: Removed LNI 192.168.201.146@tcp [ 5699.956865] Key type .llcrypt unregistered [ 5699.961628] Key type ._llcrypt unregistered [ 5731.521914] Key type ._llcrypt registered [ 5731.527457] Key type .llcrypt registered [ 5731.715097] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 5733.357346] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5733.449478] alg: No test for adler32 (adler32-zlib) [ 5734.678118] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 5734.987643] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 5736.736228] Key type lgssc registered [ 5738.091440] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5748.027433] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5748.055867] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5749.627142] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5756.040979] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5759.974476] LustreError: 172290:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5759.996775] LustreError: 172290:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 5765.089910] LustreError: 172291:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5766.631890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5767.009331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5772.119780] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5781.372945] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5789.020205] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5796.997855] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5797.463600] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5806.075647] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 5815.580436] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5820.033842] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5828.503891] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5828.553368] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5828.572571] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5828.594062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.609993] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5830.048991] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5830.089199] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5830.120079] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5830.125641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.149262] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5831.755410] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5831.787580] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5831.797820] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5831.809455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.821836] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5833.087822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.415631] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5834.429936] Lustre: Skipped 1 previous similar message [ 5834.459227] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5834.476412] Lustre: Skipped 1 previous similar message [ 5834.487107] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5834.495884] Lustre: Skipped 1 previous similar message [ 5834.499886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.510986] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5834.517326] Lustre: Skipped 1 previous similar message [ 5835.520772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.417636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.245950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.399087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.776713] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5839.791938] Lustre: Skipped 4 previous similar messages [ 5839.824266] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5839.840349] Lustre: Skipped 4 previous similar messages [ 5839.860124] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5839.869749] Lustre: Skipped 4 previous similar messages [ 5839.878404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.904751] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5839.907560] Lustre: Skipped 4 previous similar messages [ 5841.054359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.371278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.766366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.870278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.821876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.826803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.666771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.482687] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5848.490625] Lustre: Skipped 7 previous similar messages [ 5848.504216] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5848.521405] Lustre: Skipped 7 previous similar messages [ 5848.534735] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5848.538541] Lustre: Skipped 7 previous similar messages [ 5848.543691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.562111] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5848.572830] Lustre: Skipped 7 previous similar messages [ 5849.285295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.381698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.772295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.850132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.037835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.027469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.992603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.734571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.027155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.209435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.539502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.730276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.986578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.116055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.292040] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5865.298277] Lustre: Skipped 14 previous similar messages [ 5865.323794] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5865.332455] Lustre: Skipped 14 previous similar messages [ 5865.336745] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5865.339906] Lustre: Skipped 14 previous similar messages [ 5865.347152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.356819] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5865.366055] Lustre: Skipped 14 previous similar messages [ 5866.367129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.213605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.127731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.225036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.187794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.194268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.112598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.234068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.976965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.219796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.172295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.323428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.264523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.351121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.337394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.526410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.680138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.580640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.514360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.448463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.816957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.593399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.708851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.747564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.621505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.872928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.143972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.397421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.493374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.570468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.609938] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5897.614859] Lustre: Skipped 30 previous similar messages [ 5897.629981] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5897.638397] Lustre: Skipped 30 previous similar messages [ 5897.643321] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5897.649163] Lustre: Skipped 30 previous similar messages [ 5897.652895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.665593] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5897.671800] Lustre: Skipped 30 previous similar messages [ 5898.637770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.427772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.341788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.362151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.350714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.138042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.885118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.806408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.663270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.580786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.588903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.549144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.591556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.645176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.862627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.996503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.370853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.634845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.817498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.793647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.678374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.862413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.119741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.527872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.703917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.875889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.996311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.231127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.611965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.590374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.774900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.021703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.397405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.784945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.156601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.203112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.422208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.483702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.524603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.821626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.028821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.088881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.166071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.087981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.061581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.275945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.533621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.502539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.743100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.831851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.888674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.057660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.135589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.255816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.589164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.886973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.047546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.213749] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5962.221931] Lustre: Skipped 57 previous similar messages [ 5962.248681] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5962.255965] Lustre: Skipped 57 previous similar messages [ 5962.261401] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5962.266786] Lustre: Skipped 57 previous similar messages [ 5962.271076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.286100] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5962.288667] Lustre: Skipped 57 previous similar messages [ 5963.164129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.351991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.533216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.836821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.989134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.359216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.477336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.918313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.094807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.179508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.401197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.391504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.256491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.541411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.366924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.486898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.602591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.780619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.014633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.985559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.464880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.998108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.227660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.493718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.496097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.666062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.022519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.411658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.838775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.181457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.756658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.052259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.404655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.312153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.671083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.575539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.389176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.375449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.700680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.902665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.299986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.654602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.967435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.264703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.667901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.861733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.296104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.526691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.168866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.470801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.501762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.510640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.662201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.565772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.589036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.638852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.864209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.103982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.774664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.467196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.836559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.177331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.144433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.208554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.510475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.594856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.627745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.510992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.570325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.544211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.453115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.453670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.391691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.672926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.908151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.975175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.935681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.254248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.405641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.646271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.904530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.200870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.183687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.248875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.505065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.526734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.926838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.280588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.733975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.026693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.165145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.021542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.136290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.066830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.231156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.136894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.091374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.986458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.002108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.195080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.088321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.201953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.419482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.590802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.752504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.906138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.884313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.245297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.466186] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6090.476757] Lustre: Skipped 108 previous similar messages [ 6090.504886] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6090.514310] Lustre: Skipped 108 previous similar messages [ 6090.517489] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6090.523650] Lustre: Skipped 108 previous similar messages [ 6090.530588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.536279] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6090.540296] Lustre: Skipped 108 previous similar messages [ 6091.740629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.747366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.890571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.934131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.089286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.214571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.258528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.396582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.555807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.714521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.947579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.942761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.127625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.309374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.623468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.650617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.570692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.886589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.200458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.333962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.442620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.391253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.514634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.510559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.683323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.548715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.760216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.746550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.693420] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.679902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.812357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.787332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.056710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.190753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.258576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.384264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.594931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.852934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.981213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.894412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.792921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.743135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.846242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.663502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.674635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.705927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.521804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.689234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.103570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.607812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.764656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.974183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.100392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.562596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.886772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.147537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.283289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.257576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.316725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.412463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.695536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.752145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.023209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.797415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.038767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.282411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.368131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.380945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.384727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.489938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.564577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.612450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.899130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.939794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.883841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.030152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.023150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.180796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.202401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.385410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.578157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.875242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.787674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.012818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.074897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.149871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.460581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.488444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.558682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.146550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.388845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.807040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.294264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.478502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.698275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.853815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.108983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.756703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.072759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.232189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.142273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.275152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.262265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.247055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.220733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.208026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.008531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.753919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.751154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.655392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.821689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.061601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.193667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.068726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.091947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.977487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.930364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.028650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.863160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.047394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.032747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.130455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.157195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.069759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.472778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.498383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.668899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.909801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.963788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.149916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.183617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.163326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.437813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.275467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.438947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.612485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.648149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.849947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.198662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.631641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.740443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.844745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.083593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.446201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.575037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.594732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.975479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.412560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.934746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.375244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.525805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.604959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.034310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.062840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.130774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.054707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.105482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.305671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.216890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.637481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.062848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.330378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.293538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.248485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.347459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.457180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.634950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.863531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.427704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.661689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.547695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.695233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.873945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.974123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.124280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.084767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.195859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.255448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.320668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.349581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.473149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.300204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.142129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.024896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.905212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.174688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.225341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.147349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.054864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.105262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.335858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.378405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.691344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.888365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.119564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.349671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.355584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.440614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.294261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.296332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.161362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.397556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.664361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.790217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.021842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.910352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.073441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.156024] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.256354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.668397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.001549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.158642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.052177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.366394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.412712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.564646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.846664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.962432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.987772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.263701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.605955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.933935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.090377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.281953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.553562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.848382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.832839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.896348] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6346.904681] Lustre: Skipped 227 previous similar messages [ 6346.934486] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6346.966346] Lustre: Skipped 227 previous similar messages [ 6346.976748] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6346.985574] Lustre: Skipped 227 previous similar messages [ 6346.995308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.013596] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6347.033391] Lustre: Skipped 227 previous similar messages [ 6348.321499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.333393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.326419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.332334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.362751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.355620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.557127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.522649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.660173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.597511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.575733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.572816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.708700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6361.580177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.708473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.624890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.642682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.467464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.418370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.334274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.351209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.258303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.262431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.298512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.234182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.050827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.055195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.901226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.806960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.705594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.705631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.532140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.450973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.337643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.552690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.595611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.654185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.634405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.694455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.697163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.788905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.836450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.762466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.888625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.957818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.905493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.915769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.812279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.070595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.276152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.572860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.000479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.246457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.485686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.818827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.247282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.592445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.890957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.957134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.273130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.285926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.255255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.242841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.110502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.124381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.029838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.946516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.974307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.011891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.032876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.028776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.168898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.232623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.420627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.443138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.717574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.052890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.098641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.359557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.425497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.525267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.726392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.685903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.961639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.177640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.256141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.379295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.300541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.383726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.494933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.449328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.575939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.487255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.967275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.352349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.615861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.729867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.726845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.622831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.750122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.842702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.967617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.213105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.551879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.939151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.948788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.053557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.193984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.163948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.424570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.354147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.460158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.670594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.687416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.662663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.605267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.011321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.309424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.365200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.476164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.598595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.862158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.961829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.163236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.037243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.938404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.909427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.993438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.969283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.219719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.085266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.190742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.034372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.105600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.542167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.458122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.583995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.536481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.816130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.885180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.979566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.022674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.083357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.154923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.065602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.186282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.320792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.234178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.097702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.962827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.998265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.150562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.227981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.256504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.669543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.680212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.591496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6516.706529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.828586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.878955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.249746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.552681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6522.872160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.021777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.324681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.298656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.336345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.331310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.577706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.886940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.034465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.086838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6534.189811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.496984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.730312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.836967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.262730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.273687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.423468] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6542.630821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.589802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.631472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.726144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.694183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6547.602231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.736217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6549.722910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6550.791399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.699615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.835814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.932812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.068235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.220658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.306449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.195938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.258353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.235462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.258081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6562.076089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.144505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.118565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.226437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.174090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6567.283084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.256990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.401970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.813313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.632781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.689293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6573.620862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.529665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.372455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.221692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.068444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.972224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.741470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6579.749600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.733766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.629739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.489874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.454426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.309427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.156063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.987902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.042705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.971401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.086519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.061318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.940830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.107132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.267569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.172678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.091831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.033921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.911197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.844526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6598.808928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.783277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.846476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.849722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.836558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6603.752328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.788398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.818755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.876963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.173690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.426597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.393416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.230612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.059322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.929642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.688746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.492227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.429377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6616.456837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6617.215321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6618.108588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6619.046210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.053273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6620.961257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6621.939228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6622.829362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6624.146623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6625.268551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6626.206109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6627.441570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6628.532061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6629.603206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6630.703042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6632.116634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6633.026583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6634.366429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6635.490676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6636.659568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6637.855301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.044629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6639.816760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6640.920250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6641.995941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6643.130478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6644.082957] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.178247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6645.989240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6647.085691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6648.175566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6649.186889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6650.197288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6651.370332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6652.540434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6653.688096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6654.780623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6655.834838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6657.106730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6658.434190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6659.555783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6661.047535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6662.309308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6663.581448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6664.908860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6666.306432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6667.693308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6669.071175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6670.380341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6671.582300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6672.844701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6674.011831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6675.341648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6676.244456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6677.354252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6678.729967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6680.062902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6681.237353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6682.449927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6683.481809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6684.446812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6685.705097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6686.948893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6687.941929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6689.307097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6690.180513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6691.488856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6692.552922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6693.649484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6694.749251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6695.918935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6696.940419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6698.301889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6699.368578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6700.520272] LustreError: 172310:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6700.532061] LustreError: 172310:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6700.557724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6702.222566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6703.447903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6704.751821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6706.150904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6707.305083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6708.335336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6709.303303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6710.543992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6711.405447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6712.453431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6713.439361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6714.807592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6716.064903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6717.441939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6718.803313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6720.179723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6721.776572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6722.945349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6724.025120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6725.082048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6726.534238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6727.832504] LustreError: 172310:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6727.855135] LustreError: 172310:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6727.893785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6729.462773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6730.929708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6732.328125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6733.809082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6734.698389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6735.858312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6736.972939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6738.103308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6739.208364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6740.174361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6741.265251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6742.130686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6743.122861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6744.096989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6745.125583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6746.213688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6747.382450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6748.495766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6749.615844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6750.582306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6751.692377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6752.779897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6754.061689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.012364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6755.967078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6756.949964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6758.185072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6759.337621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6760.427891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6761.724904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6762.748717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6763.725827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6764.929650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.057583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6766.978924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6768.160989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6769.091405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6770.367098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6771.464773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6772.578051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6773.609400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6774.868060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6775.880411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6776.725528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6777.739728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6778.902433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.007535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6780.924789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6781.934158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.012130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6783.938945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6785.061848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6786.234303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6787.208826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6788.291588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6789.410820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6790.464424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6791.469849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6792.493872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6793.605266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6794.652615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6795.678526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6796.772816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6797.886156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6798.919758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6800.204458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6801.350865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6802.579330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6803.837678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6804.877744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6805.956083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6806.800525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6807.838085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6808.660104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6809.726508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6810.720060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6811.746849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6812.893748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6813.899508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6814.832237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6816.082344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6817.236939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6818.312374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6819.381513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6820.244739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6821.216829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6822.214445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6823.084213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.107234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6824.939818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6826.310665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6827.324719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6828.664083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6829.631216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6830.659772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6831.588679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6832.773831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6833.932707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6834.942151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6836.092652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6837.391601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6838.549794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6839.691225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6840.624518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6841.700800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6842.938347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6844.075285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6845.104587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6846.306260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6847.581339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6848.817537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6849.824922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.009121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6851.818736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6852.794160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6853.921578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6854.719893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6856.168654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6857.429264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6858.388141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.518119] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6859.525874] Lustre: Skipped 471 previous similar messages [ 6859.533993] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6859.551520] Lustre: Skipped 471 previous similar messages [ 6859.555956] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6859.560343] Lustre: Skipped 471 previous similar messages [ 6859.563660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6859.573147] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6859.578464] Lustre: Skipped 471 previous similar messages [ 6860.751779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6861.883931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6863.427121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6864.683414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6865.636931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6866.711588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6867.501716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6868.451106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6869.422095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6870.918472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6872.358444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6873.581600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6874.785935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6875.634865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6876.783091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6877.636171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6878.960517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6879.841153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6880.904678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6882.068428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6883.099989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.048363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6884.916920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6885.607698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6886.738867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6887.757619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6889.005099] LustreError: 172310:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6889.021426] LustreError: 172310:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6889.033174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6890.244099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6891.539380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6892.911519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.084617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6894.962150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6895.958620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6896.976749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6898.088135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6899.538407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6900.574743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6901.746686] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6902.950152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6904.530802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6905.728114] LustreError: 172310:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6905.746329] LustreError: 172310:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6905.771246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6907.149240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6908.798201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6910.022364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6911.625234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6912.807934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6914.043471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6915.753540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6917.064203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6918.424077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6919.474325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6920.627943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6921.947356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6922.969575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6923.957333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6924.782621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6925.804516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6926.677300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6927.407732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6928.603577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6929.553095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6930.584350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6931.729544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6932.506264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6933.359712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6934.417553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6935.516750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6936.909063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6937.900418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8070.115960] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8070.135892] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8070.158145] Lustre: Skipped 69 previous similar messages [ 8070.174112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8070.631920] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8071.140056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8071.155705] Lustre: Skipped 1 previous similar message [ 8076.261800] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8076.264305] Lustre: Skipped 1 previous similar message [ 8081.376544] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8081.383933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8081.393188] Lustre: Skipped 1 previous similar message [ 8081.650718] Lustre: server umount lustre-OST0000 complete [ 8086.498900] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8086.500342] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8086.509649] Lustre: Skipped 1 previous similar message [ 8092.167191] Lustre: server umount lustre-MDT0000 complete [ 8096.713676] LustreError: 173027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789143694 with bad export cookie 16867884299418544037 [ 8096.721898] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8096.726554] LustreError: 173027:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8097.241780] Lustre: server umount lustre-MDT0001 complete [ 8111.380686] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 8114.919351] Key type lgssc unregistered [ 8115.296844] LNet: 227614:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8115.308334] LNetError: 227614:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8115.328708] LNet: Removed LNI 192.168.201.146@tcp [ 8116.579584] Key type .llcrypt unregistered [ 8116.583314] Key type ._llcrypt unregistered [ 8148.769480] Key type ._llcrypt registered [ 8148.771529] Key type .llcrypt registered [ 8148.885471] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 8150.444203] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8150.681485] alg: No test for adler32 (adler32-zlib) [ 8151.907926] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 8152.119224] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 8153.760242] Key type lgssc registered [ 8155.337193] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8165.278925] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8165.312891] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8166.765611] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8171.364748] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8176.044114] LustreError: 229034:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8176.071427] LustreError: 229034:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8177.059048] LustreError: 229035:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8179.261303] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8179.593323] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8182.958420] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8190.352891] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8197.016286] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8205.521134] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8205.971088] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8212.114236] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49664 to 0x280000401:49695) [ 8213.970296] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8222.898793] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8228.080071] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8230.228622] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8230.250149] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8230.260296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8230.278366] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8230.282918] LustreError: 229054:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8234.977947] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8234.994404] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8235.032635] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8238.058905] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8238.066630] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8238.087227] Lustre: Skipped 2 previous similar messages [ 8240.252785] Lustre: lustre-OST0000: Not available for connect from 192.168.201.46@tcp (stopping) [ 8243.177309] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8243.190319] Lustre: Skipped 1 previous similar message [ 8247.264458] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8247.604037] Lustre: server umount lustre-OST0000 complete [ 8250.310131] LustreError: 229054:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8283.253042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8283.758324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8291.739189] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8293.995146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8294.009062] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8298.094079] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8298.108870] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8300.096982] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8301.940954] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8308.081564] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9dac87be0000.ost_server_uuid 50 [ 8309.595566] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9dac87be0000.ost_server_uuid in FULL state after 0 sec [ 8313.827784] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8313.828566] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8313.832642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8313.866899] Lustre: Skipped 2 previous similar messages [ 8319.365931] Lustre: server umount lustre-OST0000 complete [ 8328.161757] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8328.168720] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8328.183948] Lustre: Skipped 1 previous similar message [ 8328.193511] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8328.199115] Lustre: Skipped 2 previous similar messages [ 8330.839629] Lustre: server umount lustre-MDT0000 complete [ 8334.316662] LustreError: 229030:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8334.337387] LustreError: 229030:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8334.530747] LustreError: 229015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789143932 with bad export cookie 151011217051546641 [ 8334.544349] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8334.545393] LustreError: 229015:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8334.746239] Lustre: server umount lustre-MDT0001 complete [ 8347.490658] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 8350.367399] Key type lgssc unregistered [ 8350.768060] LNet: 234246:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8350.773253] LNetError: 234246:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8351.850879] LNet: Removed LNI 192.168.201.146@tcp [ 8352.774882] Key type .llcrypt unregistered [ 8352.776764] Key type ._llcrypt unregistered [ 8387.105732] Key type ._llcrypt registered [ 8387.108979] Key type .llcrypt registered [ 8387.210642] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 8388.383497] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8388.427307] alg: No test for adler32 (adler32-zlib) [ 8389.455626] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 8389.696082] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 8391.360220] Key type lgssc registered [ 8393.165571] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8404.009480] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8404.026395] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8405.560602] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8411.340833] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8415.717307] LustreError: 235978:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8415.753433] LustreError: 235978:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 8420.845066] LustreError: 235979:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8424.427092] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8425.235901] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8431.576094] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8435.716377] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8443.358800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8443.767431] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8450.938337] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49727) [ 8451.328251] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8462.702744] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8463.026421] Lustre: lustre-OST0001: new disk, initializing [ 8463.036819] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8463.051592] Lustre: 238291:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8463.241872] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8470.185768] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8470.191150] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8470.299762] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8470.873527] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8482.775672] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8489.294388] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8506.346989] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8506.353429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8506.357867] Lustre: Skipped 1 previous similar message [ 8506.387605] Lustre: Skipped 2 previous similar messages [ 8511.020406] Lustre: server umount lustre-MDT0000 complete [ 8511.467232] LustreError: 235974:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8511.485430] LustreError: 235974:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 8515.310483] LustreError: 235957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789144113 with bad export cookie 8548441121245105912 [ 8515.314055] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8515.327261] LustreError: 235957:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8515.659307] Lustre: server umount lustre-MDT0001 complete [ 8530.704822] Lustre: server umount lustre-OST0000 complete [ 8532.774431] Lustre: 235239:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789144114/real 1789144114] req@ffff95f823242580 x1876053445409536/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789144130 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8532.809080] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8532.823030] Lustre: Skipped 2 previous similar messages [ 8535.262446] Lustre: server umount lustre-OST0001 complete [ 8560.894558] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 8573.220725] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8573.887405] LustreError: 241395:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8574.079373] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8579.554652] LustreError: 241396:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8579.574523] LustreError: 241396:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8580.170427] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8588.771127] LustreError: 241395:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8588.796502] LustreError: 241395:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [ 8590.696562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8591.268351] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8596.616458] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8600.055570] Lustre: 242538:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8608.263157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8608.807240] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8617.017039] LustreError: 242893:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8617.029029] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:129) [ 8617.109099] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8619.065796] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49759) [ 8627.792508] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8633.362744] Lustre: mylustre-OST0001: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:3 to 0x2c0000400:33) [ 8637.064111] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8645.585307] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8650.392737] Lustre: 244409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8663.012236] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8663.020762] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8663.053061] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8668.990943] Lustre: server umount mylustre-MDT0000 complete [ 8669.172107] LustreError: 242129:0:(ldlm_lib.c:1228:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8669.196399] LustreError: 242129:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 7 previous similar messages [ 8673.790635] LustreError: 242540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789144271 with bad export cookie 8548441121245108705 [ 8673.805817] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8673.809761] LustreError: 242540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8674.276170] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8674.295302] Lustre: Skipped 3 previous similar messages [ 8674.636788] Lustre: server umount mylustre-MDT0001 complete [ 8679.670895] Lustre: server umount mylustre-OST0000 complete [ 8684.444501] Lustre: server umount mylustre-OST0001 complete [ 8709.754451] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 8723.686426] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8724.378550] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8724.395565] Lustre: Skipped 1 previous similar message [ 8731.105280] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8734.729819] LustreError: 246717:0:(ldlm_lib.c:1228:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8734.764144] LustreError: 246717:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [ 8742.646716] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8748.363252] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8751.443496] Lustre: 247862:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8760.150405] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8767.879761] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:161) [ 8768.786716] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8770.930965] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49791) [ 8778.808583] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8784.393511] Lustre: tfs-OST0001: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:65) [ 8787.444938] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8797.311371] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8801.301666] Lustre: 249734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8814.562971] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8814.571683] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8814.588216] Lustre: Skipped 1 previous similar message [ 8814.599082] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8814.605895] Lustre: Skipped 4 previous similar messages [ 8816.872330] Lustre: server umount tfs-MDT0000 complete [ 8821.584584] LustreError: 246693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789144419 with bad export cookie 8548441121245110882 [ 8821.588661] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8821.595794] LustreError: 246693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8822.071257] Lustre: server umount tfs-MDT0001 complete [ 8836.953859] Lustre: server umount tfs-OST0000 complete [ 8841.184356] Lustre: 235239:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789144423/real 1789144423] req@ffff95f8283d70c0 x1876053445533696/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789144439 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8841.240922] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8841.264446] Lustre: Skipped 3 previous similar messages [ 8842.505437] Lustre: server umount tfs-OST0001 complete [ 8870.076827] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 8884.579795] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8885.346173] LustreError: 251987:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8885.382622] LustreError: 251987:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 10 previous similar messages [ 8885.506430] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8885.508955] Lustre: Skipped 3 previous similar messages [ 8891.839862] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8903.830437] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8910.414121] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8914.552159] Lustre: 253129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8923.537730] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8929.135635] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:99 to 0x280000400:193) [ 8930.179396] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49697 to 0x280000401:49823) [ 8931.787235] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8941.158870] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8946.714480] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:35 to 0x2c0000400:97) [ 8949.020320] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 8958.695778] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8963.836668] Lustre: 255000:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8973.236574] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 12:36:09 (1789144569) [ 8991.208211] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8991.219566] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8991.237895] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8991.245225] Lustre: Skipped 4 previous similar messages [ 8998.055045] Lustre: server umount lustre-MDT0000 complete [ 9004.077920] LustreError: 255002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789144602 with bad export cookie 8548441121245112800 [ 9004.085524] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9004.094949] LustreError: 255002:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9008.121779] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9008.134167] Lustre: Skipped 5 previous similar messages [ 9010.710389] Lustre: server umount lustre-MDT0001 complete [ 9021.496861] Lustre: server umount lustre-OST0000 complete [ 9041.376210] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9041.600830] Lustre: server umount lustre-OST0001 complete [ 9052.115566] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 9066.633608] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 9109.605597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9110.021928] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9110.075954] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9110.235444] Lustre: lustre-MDT0000: new disk, initializing [ 9110.374460] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 9110.385870] Lustre: Skipped 3 previous similar messages [ 9110.410750] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9118.487517] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9130.689705] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9130.908687] Lustre: 258213:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9131.017583] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9131.023828] Lustre: Skipped 1 previous similar message [ 9131.149379] Lustre: lustre-MDT0001: new disk, initializing [ 9131.265371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9131.278447] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9136.242666] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9148.307366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9148.653717] Lustre: lustre-OST0000: new disk, initializing [ 9148.658521] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9148.665274] Lustre: 259797:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9150.280828] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9150.291471] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9150.458519] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9155.928694] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9168.583905] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9168.798839] Lustre: lustre-OST0001: new disk, initializing [ 9168.803739] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 9168.810635] Lustre: 260671:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9170.337581] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 9170.353818] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 9170.579866] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9177.916074] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9193.636188] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 12:39:49 (1789144789) [ 9211.556990] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 12:40:08 (1789144808) [ 9227.234402] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9227.235850] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9227.244992] LustreError: Skipped 1 previous similar message [ 9227.254407] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9227.266199] Lustre: Skipped 6 previous similar messages [ 9227.285139] Lustre: Skipped 3 previous similar messages [ 9233.664465] Lustre: server umount lustre-OST0000 complete [ 9234.935373] LustreError: 259779:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9234.960210] LustreError: 259779:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 23 previous similar messages [ 9239.010143] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9244.894779] Lustre: server umount lustre-MDT0000 complete [ 9250.270496] LustreError: 258203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789144848 with bad export cookie 8548441121245114578 [ 9250.283756] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9250.288057] LustreError: 258203:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9250.312134] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9250.329382] Lustre: Skipped 6 previous similar messages [ 9257.169294] Lustre: server umount lustre-MDT0001 complete [ 9271.739985] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 9346.966649] Lustre: server umount lustre-OST0001 complete [ 9357.411148] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [ 9378.029585] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 9427.449783] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9427.782806] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9427.835859] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9427.964729] Lustre: lustre-MDT0000: new disk, initializing [ 9428.081583] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9428.092716] Lustre: Skipped 3 previous similar messages [ 9428.146125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9432.985760] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9445.982693] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9446.219748] Lustre: 265519:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9446.356754] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9446.368548] Lustre: Skipped 1 previous similar message [ 9446.523413] Lustre: lustre-MDT0001: new disk, initializing [ 9446.744898] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9446.771725] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9452.407394] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9465.248981] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9472.479592] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9480.805759] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9481.166986] Lustre: lustre-OST0000: new disk, initializing [ 9481.172037] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9481.185317] Lustre: 267493:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9482.838587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9482.857822] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9483.030615] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9489.459652] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9501.057196] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9513.960632] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9513.976390] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9513.991719] Lustre: Skipped 4 previous similar messages [ 9513.999587] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9514.009575] Lustre: Skipped 2 previous similar messages [ 9516.244518] Lustre: server umount lustre-OST0000 complete [ 9525.933441] Lustre: server umount lustre-MDT0000 complete [ 9530.537902] LustreError: 265513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789145128 with bad export cookie 8548441121245116048 [ 9530.541195] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9530.565702] LustreError: 265513:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9530.946933] Lustre: server umount lustre-MDT0001 complete [ 9544.735714] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 9549.043199] Key type lgssc unregistered [ 9549.432689] LNet: 269256:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9549.446581] LNetError: 269256:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9549.466799] LNet: Removed LNI 192.168.201.146@tcp [ 9550.342154] Key type .llcrypt unregistered [ 9550.344670] Key type ._llcrypt unregistered [ 9577.388158] Key type ._llcrypt registered [ 9577.396322] Key type .llcrypt registered [ 9577.491739] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 9578.424703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9578.497147] alg: No test for adler32 (adler32-zlib) [ 9579.773911] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 9580.036657] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 9581.744204] Key type lgssc registered [ 9583.092637] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9594.240819] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9594.269050] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9595.871392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9600.639684] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9605.091758] LustreError: 270719:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9605.099871] LustreError: 270719:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [ 9610.221248] LustreError: 270718:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9611.149480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9611.578812] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9616.989320] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9625.304562] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9631.934674] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9638.274775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9638.604255] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9640.710975] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9644.778728] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9653.255354] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9654.092273] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9656.290949] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9656.317142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9657.383050] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9657.402935] Lustre: Skipped 1 previous similar message [ 9662.278662] Lustre: server umount lustre-OST0000 complete [ 9667.555076] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9667.561849] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9667.571379] Lustre: Skipped 1 previous similar message [ 9667.587570] Lustre: Skipped 1 previous similar message [ 9672.626147] Lustre: server umount lustre-MDT0000 complete [ 9672.677296] LustreError: 272201:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9677.382577] LustreError: 270700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789145275 with bad export cookie 7199428255875105029 [ 9677.394128] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9677.399965] LustreError: 270700:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9677.800898] Lustre: server umount lustre-MDT0001 complete [ 9689.884864] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 9692.550228] Key type lgssc unregistered [ 9692.791971] LNet: 274306:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9692.798916] LNetError: 274306:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9692.809257] LNet: Removed LNI 192.168.201.146@tcp [ 9693.807733] Key type .llcrypt unregistered [ 9693.809901] Key type ._llcrypt unregistered [ 9712.390244] Key type ._llcrypt registered [ 9712.392793] Key type .llcrypt registered [ 9712.531430] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 9714.031682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9714.063381] alg: No test for adler32 (adler32-zlib) [ 9715.765313] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 9716.240124] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 9718.016508] Key type lgssc registered [ 9720.105628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9732.092102] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9732.109192] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9747.616666] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9752.736432] LustreError: 275502:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.146@tcp: failed processing log, type 4: rc = -110 [ 9779.489876] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9787.110664] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [ 9794.960523] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9797.426180] Lustre: server umount lustre-OST0000 complete [ 9815.609423] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [ 9818.996305] Key type lgssc unregistered [ 9819.264730] LNet: 277139:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9819.275567] LNetError: 277139:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9819.305066] LNet: Removed LNI 192.168.201.146@tcp [ 9820.245678] Key type .llcrypt unregistered [ 9820.254090] Key type ._llcrypt unregistered [ 9864.256738] Key type ._llcrypt registered [ 9864.258925] Key type .llcrypt registered [ 9864.370383] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [ 9865.275703] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9865.378204] alg: No test for adler32 (adler32-zlib) [ 9866.569890] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [ 9866.843556] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [ 9868.608204] Key type lgssc registered [ 9870.190561] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9884.779904] loop0: detected capacity change from 0 to 204800000 [ 9885.042616] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9885.054776] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9885.418223] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9886.365983] loop0: detected capacity change from 0 to 204800000 [ 9886.581638] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9886.590981] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9886.957483] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9887.991023] loop0: detected capacity change from 0 to 204800000 [ 9888.219028] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9888.231770] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9888.325786] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9889.405387] loop0: detected capacity change from 0 to 204800000 [ 9889.600794] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9891.119655] loop0: detected capacity change from 0 to 204800000 [ 9891.148872] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9895.835449] loop0: detected capacity change from 0 to 204800000 [ 9895.865458] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9900.425589] loop0: detected capacity change from 0 to 204800000 [ 9900.469120] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9905.283968] loop0: detected capacity change from 0 to 204800000 [ 9905.307116] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9910.727606] loop0: detected capacity change from 0 to 204800000 [ 9910.832052] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9910.890042] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9911.015379] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9911.038885] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9911.049632] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9911.059907] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9911.079143] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9911.095859] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9911.106907] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9911.125909] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9911.141686] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9911.156799] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9911.170350] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9911.183191] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9911.198543] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9911.208051] Lustre: 280556:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9918.096093] Lustre: server umount MGS complete [ 9919.092327] loop0: detected capacity change from 0 to 204800000 [ 9919.211460] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9919.879612] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9919.945717] Lustre: lustre-MDT0000: Aborting client recovery [ 9919.950300] LustreError: 280931:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0000-osd: get update log duration 0, retries 0, failed: rc = -108 [ 9920.952385] loop1: detected capacity change from 0 to 204800000 [ 9921.120941] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9921.247306] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9921.258653] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9921.271904] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9921.284612] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9921.298676] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9921.315769] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9921.339270] Lustre: 280988:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9921.393437] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9921.562527] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9921.646840] Lustre: lustre-MDT0001: Aborting client recovery [ 9922.676699] loop2: detected capacity change from 0 to 204800000 [ 9922.870029] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9922.945409] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9922.962934] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9922.998262] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9923.020940] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9923.045300] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9923.070960] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9923.085871] Lustre: 281066:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9923.227015] Lustre: Found index 0 for lustre-OST0000, updating log [ 9923.296950] Lustre: 281085:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9923.377652] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9923.400714] Lustre: lustre-OST0000: Aborting recovery [ 9924.579529] loop3: detected capacity change from 0 to 204800000 [ 9924.786840] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9924.893136] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9924.918407] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9924.942295] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9924.958970] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9924.966925] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9924.984775] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9924.993763] Lustre: 281151:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9925.016723] Lustre: Found index 1 for lustre-OST0001, updating log [ 9925.086609] Lustre: 281158:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9925.246314] Lustre: lustre-OST0001: Aborting recovery [ 9929.324665] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9929.343175] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9929.353091] LustreError: 280932:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9929.372753] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9929.406719] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9931.385508] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9931.399057] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9934.581158] LustreError: 280899:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 2695us: rc = -2 [ 9935.194231] LustreError: 280900:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 610us: rc = -2 [ 9935.236757] LustreError: 280900:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 9952.225587] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9952.230260] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9952.233044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9952.310093] Lustre: Skipped 3 previous similar messages [ 9955.196813] Lustre: server umount lustre-MDT0000 complete [ 9956.407209] LustreError: 280882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789145554 with bad export cookie 17997756266253987300 [ 9956.412623] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9956.420048] LustreError: 280882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9957.346858] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9957.349871] LustreError: 280896:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9957.366184] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9957.400310] LustreError: 280896:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [ 9957.407879] Lustre: Skipped 3 previous similar messages [ 9961.377065] LustreError: 280896:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9961.377692] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9961.406359] Lustre: Skipped 2 previous similar messages [ 9962.985165] Lustre: server umount lustre-MDT0001 complete [ 9978.336140] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9978.591407] Lustre: server umount lustre-OST0000 complete [ 9994.208172] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9994.618021] Lustre: server umount lustre-OST0001 complete [10005.490666] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 12:53:22 (1789145602) [10042.973977] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [10054.777357] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [10095.949895] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10096.323966] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10096.347392] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10096.426420] Lustre: lustre-MDT0000: new disk, initializing [10096.496294] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10096.505235] Lustre: Skipped 1 previous similar message [10096.524263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10102.147413] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10114.190568] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10114.338233] Lustre: 285242:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10114.425344] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10114.429813] Lustre: Skipped 1 previous similar message [10114.505346] Lustre: lustre-MDT0001: new disk, initializing [10114.632210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10114.719106] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10114.740407] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10119.285641] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10129.655472] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10136.611512] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10144.188859] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10144.666359] Lustre: lustre-OST0000: new disk, initializing [10144.671566] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10144.680476] Lustre: 287210:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10144.784953] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10150.859860] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10152.009531] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10152.014074] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10152.099612] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10161.164859] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10166.575372] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10180.322636] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10188.660873] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10193.323502] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10234.338125] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10234.341116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10234.347504] LustreError: Skipped 1 previous similar message [10234.358404] Lustre: Skipped 1 previous similar message [10234.361675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10236.833796] Lustre: server umount lustre-OST0000 complete [10242.530748] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10242.541353] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10242.552704] Lustre: Skipped 1 previous similar message [10242.570826] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10242.576905] Lustre: Skipped 1 previous similar message [10247.251198] Lustre: server umount lustre-MDT0000 complete [10249.706481] LustreError: 285252:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10249.725835] LustreError: 285252:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [10251.337354] LustreError: 285233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789145849 with bad export cookie 17997756266253990009 [10251.341938] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10251.349343] LustreError: 285233:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10251.715942] Lustre: server umount lustre-MDT0001 complete [10262.958921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10269.341425] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10275.899613] Lustre: server umount MGS complete [10289.435657] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10289.836384] LustreError: 291169:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10289.864581] LustreError: 291182:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10289.951410] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10295.279803] LustreError: 291170:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10295.350658] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10304.482023] LustreError: 291170:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10304.510139] LustreError: 291170:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10305.407046] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10311.040138] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10320.691933] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10328.575958] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10337.118792] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10337.703955] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10337.720071] Lustre: Skipped 1 previous similar message [10345.277499] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10354.569779] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10392.037920] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10392.052664] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10392.073090] Lustre: Skipped 1 previous similar message [10392.080893] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10392.085970] Lustre: Skipped 2 previous similar messages [10404.322485] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10404.564542] Lustre: server umount lustre-OST0000 complete [10408.419299] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10408.429394] LustreError: Skipped 1 previous similar message [10408.439770] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10408.462624] Lustre: Skipped 1 previous similar message [10408.465337] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10408.474921] Lustre: Skipped 4 previous similar messages [10414.600583] Lustre: server umount lustre-MDT0000 complete [10415.587823] LustreError: 291170:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10415.607143] LustreError: 291170:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10418.844945] LustreError: 291148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789146016 with bad export cookie 17997756266253991850 [10418.860522] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10418.862387] LustreError: 291148:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10419.313450] Lustre: server umount lustre-MDT0001 complete [10431.950185] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [10435.512756] Key type lgssc unregistered [10435.971523] LNet: 295125:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10435.995233] LNetError: 295125:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10437.031458] LNet: Removed LNI 192.168.201.146@tcp [10438.107139] Key type .llcrypt unregistered [10438.110702] Key type ._llcrypt unregistered [10481.212696] Key type ._llcrypt registered [10481.215284] Key type .llcrypt registered [10481.315415] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [10497.082524] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [10498.281592] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10498.379712] alg: No test for adler32 (adler32-zlib) [10499.345765] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [10499.553346] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [10501.288173] Key type lgssc registered [10502.696654] Lustre: Echo OBD driver; http://www.lustre.org/ [10543.115417] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10543.133964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10544.382371] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10544.436488] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10544.546274] Lustre: lustre-MDT0000: new disk, initializing [10544.634194] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10544.673264] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10549.495524] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10560.580315] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10560.756565] Lustre: 299098:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10560.837459] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10560.845761] Lustre: Skipped 1 previous similar message [10560.978351] Lustre: lustre-MDT0001: new disk, initializing [10561.058024] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10561.103256] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10561.120808] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10566.335282] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10577.400197] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10584.040604] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10591.427729] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10591.677872] Lustre: lustre-OST0000: new disk, initializing [10591.680720] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10591.685745] Lustre: 301066:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10591.770164] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10598.488465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10598.499789] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10598.606647] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10598.787741] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10609.376783] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10617.292587] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10631.561906] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10639.190461] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10649.877164] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10696.161953] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10696.162770] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10696.178063] LustreError: Skipped 1 previous similar message [10696.203731] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10696.212110] Lustre: Skipped 1 previous similar message [10699.249269] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10700.663444] Lustre: server umount lustre-OST0000 complete [10709.473569] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10709.475055] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10709.488856] Lustre: Skipped 1 previous similar message [10709.493595] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10709.493616] Lustre: Skipped 1 previous similar message [10711.014826] Lustre: server umount lustre-MDT0000 complete [10716.336704] LustreError: 299090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789146314 with bad export cookie 17406359910097703390 [10716.350496] LustreError: 299090:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10716.356715] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10717.053676] Lustre: server umount lustre-MDT0001 complete [10727.196650] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10732.764531] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10738.275077] Lustre: server umount MGS complete [10752.465952] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10752.967179] LustreError: 305054:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10752.977622] LustreError: 305054:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [10752.983720] LustreError: 305067:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10753.090720] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10758.122750] LustreError: 305054:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10758.158310] LustreError: 305054:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10758.633392] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10762.234148] LustreError: 305073:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10767.344417] LustreError: 305054:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10768.909487] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10769.260842] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10774.065199] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10783.726142] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10790.039315] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10798.442959] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10798.982799] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10807.292500] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [10817.707487] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10853.352651] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10853.355341] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10853.382448] Lustre: Skipped 1 previous similar message [10853.383776] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10853.383784] Lustre: Skipped 1 previous similar message [10858.011149] Lustre: server umount lustre-OST0000 complete [10863.597380] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10863.604445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10863.626628] Lustre: Skipped 2 previous similar messages [10863.667497] Lustre: Skipped 2 previous similar messages [10869.756624] Lustre: server umount lustre-MDT0000 complete [10873.828474] LustreError: 307481:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10873.851962] LustreError: 307481:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [10875.588264] LustreError: 307784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789146473 with bad export cookie 17406359910097705322 [10875.590382] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10875.606670] LustreError: 307784:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10876.236951] Lustre: server umount lustre-MDT0001 complete [10888.815840] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [10891.716160] Key type lgssc unregistered [10892.091858] LNet: 309004:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10892.098242] LNetError: 309004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10892.121526] LNet: Removed LNI 192.168.201.146@tcp [10892.984627] Key type .llcrypt unregistered [10892.986784] Key type ._llcrypt unregistered [10916.401166] Key type ._llcrypt registered [10916.403335] Key type .llcrypt registered [10916.568979] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [10932.371586] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [10933.669471] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10933.715979] alg: No test for adler32 (adler32-zlib) [10934.805542] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [10935.143707] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [10936.857221] Key type lgssc registered [10938.170492] Lustre: Echo OBD driver; http://www.lustre.org/ [10966.993316] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [10985.080061] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10985.124628] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10986.664592] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10986.764487] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10986.986302] Lustre: lustre-MDT0000: new disk, initializing [10987.142352] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10987.168134] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10993.605950] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [11006.327748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11006.588519] Lustre: lustre-OST0000: new disk, initializing [11006.592894] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11006.602764] Lustre: Skipped 1 previous similar message [11006.614679] Lustre: 313059:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11006.733565] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11014.531868] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [11016.849295] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [11016.870647] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [11016.907209] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [11946.468022] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11946.489552] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11949.006568] Lustre: server umount lustre-MDT0000 complete [11954.728029] LustreError: 312243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789147552 with bad export cookie 10875122096570571377 [11954.734384] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11954.748582] LustreError: 312243:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11954.973803] Lustre: server umount lustre-OST0000 complete [11987.300958] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [12000.158872] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [12040.580235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12040.820781] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12040.839218] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12040.897406] Lustre: lustre-MDT0000: new disk, initializing [12040.970642] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12040.988732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12046.392400] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12058.317791] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12058.477257] Lustre: 317407:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12058.571811] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12058.580038] Lustre: Skipped 1 previous similar message [12058.703298] Lustre: lustre-MDT0001: new disk, initializing [12058.858264] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12058.918932] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12058.941120] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12065.174322] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12076.786686] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12085.457525] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12092.775347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12093.142368] Lustre: lustre-OST0000: new disk, initializing [12093.146241] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12093.156372] Lustre: 319377:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12093.304276] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12100.219752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12100.237680] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12100.376643] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12100.683665] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12111.438412] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12116.558343] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12116.841770] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12121.425655] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12121.689067] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12126.181028] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12126.190392] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12126.202472] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12126.217467] Lustre: Skipped 1 previous similar message [12130.068397] Lustre: server umount lustre-OST0000 complete [12135.914192] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12135.927496] LustreError: Skipped 1 previous similar message [12135.934307] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12135.948603] Lustre: Skipped 1 previous similar message [12135.958431] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12140.795950] Lustre: server umount lustre-MDT0000 complete [12145.582575] LustreError: 318201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789147743 with bad export cookie 10875122096573512196 [12145.596289] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12145.598105] LustreError: 318201:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12146.003491] Lustre: server umount lustre-MDT0001 complete [12150.301389] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12153.338385] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 13:29:09 (1789147749) [12161.632128] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12162.213337] LustreError: 321691:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12162.229385] LustreError: 321691:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12162.287281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12167.366832] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12167.657430] LustreError: 321692:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12172.770639] LustreError: 321691:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12177.374813] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12177.799399] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12182.373096] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12190.579632] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12197.552663] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12204.671398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12205.030100] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12211.503088] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12221.454741] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12229.235280] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12229.344185] Lustre: lustre-OST0001: new disk, initializing [12229.347146] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12229.351982] Lustre: 324528:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12229.446748] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12235.940720] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12238.428096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12238.434596] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12238.487717] LustreError: 324989:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12238.495426] LustreError: 324989:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12239.525384] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12239.532972] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12240.560598] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12240.575267] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [12242.616564] LustreError: 324989:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12242.635187] LustreError: 324989:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12246.302164] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12246.674898] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12246.787730] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12246.798629] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [12252.719428] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9dac87be5000.ost_server_uuid 50 [12254.362787] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9dac87be5000.ost_server_uuid in FULL state after 0 sec [12255.075862] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12255.080945] LustreError: 324993:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 16 previous similar messages [12261.222831] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9dac87be5000.ost_server_uuid 50 [12263.047638] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9dac87be5000.ost_server_uuid in FULL state after 0 sec [12265.544843] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12279.788754] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12279.790318] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12279.812080] Lustre: Skipped 2 previous similar messages [12279.822268] Lustre: Skipped 3 previous similar messages [12284.798675] Lustre: server umount lustre-OST0001 complete [12284.912324] LustreError: 323514:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12284.927248] LustreError: 323514:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12290.031438] LustreError: 326063:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12290.051581] LustreError: 326063:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12294.117438] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12294.117863] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12294.133953] LustreError: Skipped 1 previous similar message [12294.135979] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12294.153091] Lustre: Skipped 1 previous similar message [12294.157993] Lustre: Skipped 1 previous similar message [12297.346649] Lustre: server umount lustre-OST0000 complete [12305.379229] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12305.387110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12305.389166] Lustre: Skipped 1 previous similar message [12305.410288] Lustre: Skipped 4 previous similar messages [12308.622057] Lustre: server umount lustre-MDT0000 complete [12310.501682] LustreError: 325911:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12310.511984] LustreError: 325911:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12313.168895] LustreError: 324527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789147911 with bad export cookie 10875122096573513071 [12313.171254] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12313.179616] LustreError: 324527:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12313.433631] Lustre: server umount lustre-MDT0001 complete [12327.861591] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [12330.868244] Key type lgssc unregistered [12331.161298] LNet: 327222:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12331.169754] LNetError: 327222:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12332.197952] LNet: Removed LNI 192.168.201.146@tcp [12333.381696] Key type .llcrypt unregistered [12333.385410] Key type ._llcrypt unregistered [12360.823642] Key type ._llcrypt registered [12360.826897] Key type .llcrypt registered [12360.934629] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [12361.893450] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12361.912952] alg: No test for adler32 (adler32-zlib) [12363.084150] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [12363.465570] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [12365.168300] Key type lgssc registered [12366.572472] Lustre: Echo OBD driver; http://www.lustre.org/ [12374.944572] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12374.977507] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12376.442598] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12381.665251] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12386.789691] LustreError: 328646:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12386.805972] LustreError: 328646:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12390.090838] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12391.913050] LustreError: 328647:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12397.035259] LustreError: 328646:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12398.893409] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12399.363912] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12404.780458] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12415.745844] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12416.211738] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12421.370323] LustreError: 330328:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12421.429488] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12423.819854] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12429.794319] LustreError: 330583:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12429.807258] LustreError: 330583:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12432.964359] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12440.046927] LustreError: 330329:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12440.075497] LustreError: 330329:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12442.955500] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12443.586676] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12448.800828] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12453.788987] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12463.523991] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12471.079283] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12471.356570] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12576.747346] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12576.753508] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12576.785685] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12576.795694] Lustre: Skipped 1 previous similar message [12581.858318] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12586.991542] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12587.003045] Lustre: Skipped 2 previous similar messages [12590.048985] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12590.319118] Lustre: server umount lustre-OST0001 complete [12592.113094] LustreError: 330328:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12592.144411] LustreError: 330328:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [12598.755253] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12598.765698] LustreError: Skipped 1 previous similar message [12598.769146] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12598.784681] Lustre: Skipped 1 previous similar message [12598.806656] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12600.816069] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12600.821574] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12603.979773] Lustre: server umount lustre-OST0000 complete [12612.587622] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12612.621498] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12612.635949] Lustre: Skipped 4 previous similar messages [12622.820429] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12622.834407] Lustre: Skipped 3 previous similar messages [12624.354188] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12624.810842] Lustre: server umount lustre-MDT0000 complete [12627.938702] LustreError: 328647:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12627.964557] LustreError: 328647:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [12629.441765] LustreError: 328625:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789148227 with bad export cookie 12453150113341949708 [12629.455191] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12630.027192] Lustre: server umount lustre-MDT0001 complete [12644.407337] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [12647.793415] Key type lgssc unregistered [12648.278963] LNet: 333599:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12648.290858] LNetError: 333599:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12648.306802] LNet: Removed LNI 192.168.201.146@tcp [12649.208873] Key type .llcrypt unregistered [12649.213561] Key type ._llcrypt unregistered [12711.805829] Key type ._llcrypt registered [12711.809367] Key type .llcrypt registered [12711.962306] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [12713.205071] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12713.233871] alg: No test for adler32 (adler32-zlib) [12714.491318] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [12714.735353] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [12716.472350] Key type lgssc registered [12717.822084] Lustre: Echo OBD driver; http://www.lustre.org/ [12728.277695] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12728.297274] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12729.995804] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12736.415977] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12739.557786] LustreError: 335893:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12739.594594] LustreError: 335893:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [12744.673289] LustreError: 335894:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12748.271824] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12749.016092] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12754.666738] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12757.997585] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12766.208344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12766.722379] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12773.900212] LustreError: 337381:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12773.931242] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12774.912985] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12780.005121] LustreError: 337748:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12780.035686] LustreError: 337748:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12783.036687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12783.213750] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12788.735899] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12790.737881] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12801.766867] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12807.627910] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12824.564717] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12824.568473] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12824.597399] Lustre: Skipped 2 previous similar messages [12824.620995] Lustre: Skipped 3 previous similar messages [12827.422848] Lustre: server umount lustre-MDT0000 complete [12829.682609] LustreError: 338664:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12829.714498] LustreError: 338664:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [12831.882050] LustreError: 335875:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789148429 with bad export cookie 3625406185411891000 [12831.885774] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12831.897355] LustreError: 335875:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12832.278933] Lustre: server umount lustre-MDT0001 complete [12847.182560] Lustre: server umount lustre-OST0000 complete [12861.249807] Lustre: server umount lustre-OST0001 complete [12888.146693] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [12905.075056] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12905.573187] LustreError: 341161:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12905.649507] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12912.603299] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12923.190417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12923.515478] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12928.170377] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12932.081215] Lustre: 342303:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12939.692344] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12939.967408] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12946.516828] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12948.146751] LustreError: 342656:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12948.177182] LustreError: 342656:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 3 previous similar messages [12951.234240] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12955.666687] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12961.282371] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12961.986844] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [12970.206068] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12975.169805] Lustre: 344175:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12979.681918] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12979.692964] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12979.708722] Lustre: Skipped 1 previous similar message [12979.715637] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12981.732270] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12981.733347] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12981.744736] Lustre: Skipped 2 previous similar messages [12981.752609] Lustre: Skipped 3 previous similar messages [12986.039845] Lustre: server umount lustre-MDT0000 complete [12986.866375] LustreError: 341156:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12986.897895] LustreError: 341156:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 5 previous similar messages [12991.179227] LustreError: 341141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789148589 with bad export cookie 3625406185411892694 [12991.182807] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12991.198250] LustreError: 341141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [12992.015610] Lustre: server umount lustre-MDT0001 complete [13006.965743] Lustre: server umount lustre-OST0000 complete [13007.268325] Lustre: 335156:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1789148589/real 1789148589] req@ffff95f803ae4b40 x1876057980522368/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1789148605 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [13007.304579] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13011.530246] Lustre: server umount lustre-OST0001 complete [13015.496797] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [13017.422604] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [13019.626807] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 13:43:36 (1789148616) [13046.109303] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [13055.746734] loop0: detected capacity change from 0 to 1099511627776 [13056.373820] SGI XFS with ACLs, security attributes, quota, no debug enabled [13056.378885] *** VALIDATE xfs *** [13056.387667] XFS (loop0): Mounting V5 Filesystem [13056.421389] XFS (loop0): Ending clean mount [13057.651685] loop1: detected capacity change from 0 to 18691697672192 [13177.456903] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [13188.964362] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [13229.697920] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13230.188847] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [13230.219233] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [13230.300912] Lustre: lustre-MDT0000: new disk, initializing [13230.383099] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13230.390667] Lustre: Skipped 1 previous similar message [13230.426798] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [13236.476167] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13248.497907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13248.607521] Lustre: 349314:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13248.667312] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [13248.671468] Lustre: Skipped 1 previous similar message [13248.732560] Lustre: lustre-MDT0001: new disk, initializing [13248.831303] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [13248.838049] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13254.514457] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13264.422500] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13272.272599] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13282.125330] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13282.469773] Lustre: lustre-OST0000: new disk, initializing [13282.475440] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13282.490350] Lustre: 351285:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13283.941593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13283.948748] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13284.083147] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13290.765771] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13302.413604] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13308.334850] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13308.541653] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13313.984326] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13314.250552] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13320.174211] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13320.176367] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13320.197943] LustreError: Skipped 1 previous similar message [13320.206056] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13320.220823] Lustre: Skipped 1 previous similar message [13320.256634] Lustre: Skipped 1 previous similar message [13323.253414] Lustre: server umount lustre-OST0000 complete [13330.401985] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13330.409784] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13330.421240] Lustre: Skipped 1 previous similar message [13330.446610] Lustre: Skipped 1 previous similar message [13334.325857] Lustre: server umount lustre-MDT0000 complete [13335.527925] LustreError: 349322:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13335.554876] LustreError: 349322:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [13339.359996] LustreError: 350119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789148937 with bad export cookie 3625406185411894255 [13339.365610] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13339.376562] LustreError: 350119:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13339.993224] Lustre: server umount lustre-MDT0001 complete [13351.802042] XFS (loop0): Unmounting Filesystem [13357.797657] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 13:49:14 (1789148954) [13365.373214] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13366.048516] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13366.052707] Lustre: Skipped 2 previous similar messages [13370.790850] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13380.874289] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13388.209626] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13402.458119] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13410.867483] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13419.444026] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13427.054050] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13436.417180] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13447.649540] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13447.657865] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13447.677873] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13450.727479] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13453.493769] Lustre: server umount lustre-OST0000 complete [13458.406480] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13464.496131] Lustre: server umount lustre-MDT0000 complete [13466.084719] LustreError: 353842:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13466.118957] LustreError: 353842:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 6 previous similar messages [13470.999736] LustreError: 353828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789149068 with bad export cookie 3625406185411895137 [13471.011306] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13471.015869] LustreError: 353828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13471.628533] Lustre: server umount lustre-MDT0001 complete [13485.778062] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [13488.950995] Key type lgssc unregistered [13489.229650] LNet: 357482:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13489.239629] LNetError: 357482:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13489.257852] LNet: Removed LNI 192.168.201.146@tcp [13490.018158] Key type .llcrypt unregistered [13490.020881] Key type ._llcrypt unregistered [13526.610867] Key type ._llcrypt registered [13526.612199] Key type .llcrypt registered [13526.857424] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [13528.565533] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13529.218834] alg: No test for adler32 (adler32-zlib) [13530.426907] Lustre: Lustre: Build Version: 2.17.57_127_ged01a19 [13530.729947] LNet: Added LNI 192.168.201.146@tcp [8/256/0/180] [13532.512290] Key type lgssc registered [13534.067993] Lustre: Echo OBD driver; http://www.lustre.org/ [13542.623111] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13542.674190] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13544.131882] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13549.102503] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13554.668250] LustreError: 359056:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13554.692414] LustreError: 359056:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13558.074942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13558.433433] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13562.886346] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13572.156974] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13580.278085] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13588.595238] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13589.009464] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13597.121833] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13605.990819] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13614.052286] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13614.075545] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13617.817336] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.46@tcp (stopping) [13617.823441] Lustre: Skipped 1 previous similar message [13619.189667] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13622.905059] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.46@tcp (stopping) [13622.923025] Lustre: Skipped 1 previous similar message [13627.360468] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13627.918674] Lustre: server umount lustre-MDT0001 complete [13628.028928] LustreError: 359049:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13629.422782] LustreError: 361549:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13629.450963] LustreError: 361549:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 1 previous similar message [13633.143823] LustreError: 359051:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13638.272743] LustreError: 359050:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.46@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13638.303162] LustreError: 359050:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13640.822942] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13640.876465] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13640.883153] Lustre: Found index 1 for lustre-MDT0001, updating log [13640.903551] Lustre: 359041:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13641.160508] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13646.369183] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13646.890878] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13649.926239] LustreError: 362658:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13649.937978] LustreError: 362658:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13649.952666] LustreError: 359046:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13650.023750] LustreError: 362661:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.146@tcp: cfg command failed: rc = -17 [13650.028403] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13650.028403] [13953.520595] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13953.533467] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13953.547821] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13953.557965] Lustre: Skipped 2 previous similar messages [13953.564873] Lustre: Skipped 2 previous similar messages [13958.627388] LustreError: 359056:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13958.644422] LustreError: 359056:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 2 previous similar messages [13958.884530] Lustre: server umount lustre-MDT0001 complete [13970.432646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13970.558190] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13970.564445] Lustre: Found index 1 for lustre-MDT0001, updating log [13970.625575] Lustre: 359043:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13971.054234] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13976.073759] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13976.508446] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [13977.613191] LustreError: 370637:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13977.624808] LustreError: 370637:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13977.637042] LustreError: 359046:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13977.640076] LustreError: 359046:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13977.741741] LustreError: 370646:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13977.759760] LustreError: 370646:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13977.769738] LustreError: 370646:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.146@tcp: cfg command failed: rc = -17 [13977.787153] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.146@tcp [13977.787153] [14288.356685] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14288.358349] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14288.384038] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14288.410706] Lustre: Skipped 3 previous similar messages [14298.080747] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [14298.382627] Lustre: server umount lustre-MDT0001 complete [14298.610728] LustreError: 359783:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14298.627946] LustreError: 359783:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 8 previous similar messages [14313.754282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14314.108800] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14314.114555] Lustre: Found index 1 for lustre-MDT0001, updating log [14314.182894] Lustre: 359041:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14314.883926] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14320.109760] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14321.176501] LustreError: 378566:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14321.185406] LustreError: 378566:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14321.193716] LustreError: 359046:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14321.208674] LustreError: 359046:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14321.327211] LustreError: 378576:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14321.333890] LustreError: 378576:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.201.146@tcp: cfg command failed: rc = -114 [14321.338167] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14321.338167] [14321.728125] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [14647.780025] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14647.790167] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14647.799621] Lustre: Skipped 1 previous similar message [14647.804885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14647.821359] Lustre: Skipped 6 previous similar messages [14651.615131] Lustre: server umount lustre-MDT0000 complete [14652.908301] LustreError: 359049:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14652.982515] LustreError: 359049:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 12 previous similar messages [14655.394909] LustreError: 361684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789150253 with bad export cookie 17621775464485633217 [14655.405675] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14655.409905] LustreError: 361684:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14658.026677] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14658.045441] Lustre: Skipped 2 previous similar messages [14661.836123] Lustre: server umount lustre-MDT0001 complete [14667.073418] Lustre: server umount lustre-OST0000 complete [14679.268222] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [14691.642686] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [14732.452938] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 14:12:09 (1789150329) [14741.001597] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14741.455629] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14741.548905] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14741.838712] Lustre: lustre-MDT0000: new disk, initializing [14741.957276] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14742.008308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14747.481452] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [14761.569422] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14761.701662] Lustre: 388759:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14761.756974] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14761.763562] Lustre: Skipped 1 previous similar message [14761.831416] Lustre: lustre-MDT0001: new disk, initializing [14761.923032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14761.999749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14762.010337] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14767.625549] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [14782.828218] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14793.693064] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14805.252914] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14806.079230] Lustre: lustre-OST0000: new disk, initializing [14806.091310] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14806.106775] Lustre: 390728:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14806.274383] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14813.689814] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_default_debug -1 all [14815.343880] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14815.370030] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14815.458811] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14826.042984] Lustre: DEBUG MARKER: oleg146-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14835.582710] LustreError: lustre-MDT0000-osp-MDT0001: operation out_update to node 0@lo failed: rc = -107 [14835.590235] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14835.607809] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14835.611475] Lustre: Skipped 2 previous similar messages [14835.818710] Lustre: server umount lustre-MDT0000 complete [14836.205048] LustreError: 388770:0:(ldlm_lib.c:1228:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14836.228673] LustreError: 388770:0:(ldlm_lib.c:1228:target_handle_connect()) Skipped 4 previous similar messages [14840.442365] LustreError: 388749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1789150438 with bad export cookie 17621775464485635170 [14840.445693] LustreError: MGC192.168.201.146@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14840.461839] LustreError: 388749:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14840.467613] LustreError: 389565:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14840.931781] Lustre: server umount lustre-MDT0001 complete [14857.597584] Lustre: server umount lustre-OST0000 complete [14896.705597] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing set_hostid [14912.722573] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing load_modules_local [14949.176792] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14681 sec ========= 14:15:45 (1789150545) [14951.367407] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 14:15:47 (1789150547) === [14956.628747] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 14:15:52 (1789150552) === [15013.051281] Lustre: DEBUG MARKER: oleg146-server.virtnet: executing unload_modules_local [15016.237620] Key type lgssc unregistered [15016.573712] LNet: 397105:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [15016.581179] LNetError: 397105:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [15016.598337] LNet: Removed LNI 192.168.201.146@tcp [15017.813871] Key type .llcrypt unregistered [15017.816440] Key type ._llcrypt unregistered