[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 485158697 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001012] APIC: Switch to symmetric I/O mode setup [ 0.002414] x2apic enabled [ 0.003009] Switched APIC routing to physical x2apic. [ 0.004012] kvm-guest: setup PV IPIs [ 0.007000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.007000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.007022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.008013] pid_max: default: 32768 minimum: 301 [ 0.009153] LSM: Security Framework initializing [ 0.010056] Yama: becoming mindful. [ 0.011034] SELinux: Initializing. [ 0.012078] *** VALIDATE selinux *** [ 0.021537] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026571] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.027167] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.028123] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030055] *** VALIDATE tmpfs *** [ 0.031455] *** VALIDATE proc *** [ 0.033157] *** VALIDATE cgroup *** [ 0.034011] *** VALIDATE cgroup2 *** [ 0.036183] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037165] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039033] Spectre V2 : User space: Vulnerable [ 0.040010] Speculative Store Bypass: Vulnerable [ 0.042932] debug: unmapping init [mem 0xffffffffb8459000-0xffffffffb8460fff] [ 0.044906] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.045818] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.046028] ... version: 2 [ 0.047015] ... bit width: 48 [ 0.048015] ... generic registers: 4 [ 0.049019] ... value mask: 0000ffffffffffff [ 0.050013] ... max period: 00007fffffffffff [ 0.051016] ... fixed-purpose events: 3 [ 0.052015] ... event mask: 000000070000000f [ 0.053345] rcu: Hierarchical SRCU implementation. [ 0.055504] smp: Bringing up secondary CPUs ... [ 0.056601] x86: Booting SMP configuration: [ 0.057026] .... node #0, CPUs: #1 #2 #3 [ 0.060545] smp: Brought up 1 node, 4 CPUs [ 0.062014] smpboot: Max logical packages: 1 [ 0.063023] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.136024] node 0 deferred pages initialised in 71ms [ 0.141060] devtmpfs: initialized [ 0.142210] x86/mm: Memory block size: 128MB [ 0.145707] gcov: version magic: 0x41383552 [ 0.148296] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.151071] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.153345] pinctrl core: initialized pinctrl subsystem [ 0.154156] [ 0.154637] ************************************************************* [ 0.157010] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.158008] ** ** [ 0.160011] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.162010] ** ** [ 0.164010] ** This means that this kernel is built to expose internal ** [ 0.165010] ** IOMMU data structures, which may compromise security on ** [ 0.167011] ** your system. ** [ 0.169011] ** ** [ 0.171018] ** If you see this message and you are not debugging the ** [ 0.173010] ** kernel, report this immediately to your vendor! ** [ 0.174008] ** ** [ 0.175015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.177011] ************************************************************* [ 0.179650] NET: Registered protocol family 16 [ 0.181660] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.184085] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.186079] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.190152] cpuidle: using governor menu [ 0.191735] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.193628] PCI: Using configuration type 1 for base access [ 0.196246] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.206207] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.207029] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.209136] cryptd: max_cpu_qlen set to 1000 [ 0.213200] ACPI: Added _OSI(Module Device) [ 0.214035] ACPI: Added _OSI(Processor Device) [ 0.215000] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.216017] ACPI: Added _OSI(Processor Aggregator Device) [ 0.221096] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.227539] ACPI: Interpreter enabled [ 0.229065] ACPI: PM: (supports S0 S3 S4 S5) [ 0.230010] ACPI: Using IOAPIC for interrupt routing [ 0.232218] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.234507] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.244724] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.247053] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.249023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.252087] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.257500] acpiphp: Slot [2] registered [ 0.259167] acpiphp: Slot [5] registered [ 0.260207] acpiphp: Slot [6] registered [ 0.262163] acpiphp: Slot [7] registered [ 0.263170] acpiphp: Slot [8] registered [ 0.265295] acpiphp: Slot [9] registered [ 0.266150] acpiphp: Slot [10] registered [ 0.268208] acpiphp: Slot [3] registered [ 0.269179] acpiphp: Slot [4] registered [ 0.271153] acpiphp: Slot [11] registered [ 0.273121] acpiphp: Slot [12] registered [ 0.274142] acpiphp: Slot [13] registered [ 0.276125] acpiphp: Slot [14] registered [ 0.277113] acpiphp: Slot [15] registered [ 0.279104] acpiphp: Slot [16] registered [ 0.280107] acpiphp: Slot [17] registered [ 0.282115] acpiphp: Slot [18] registered [ 0.283091] acpiphp: Slot [19] registered [ 0.285127] acpiphp: Slot [20] registered [ 0.286084] acpiphp: Slot [21] registered [ 0.287120] acpiphp: Slot [22] registered [ 0.289091] acpiphp: Slot [23] registered [ 0.290116] acpiphp: Slot [24] registered [ 0.292104] acpiphp: Slot [25] registered [ 0.293131] acpiphp: Slot [26] registered [ 0.294174] acpiphp: Slot [27] registered [ 0.296096] acpiphp: Slot [28] registered [ 0.297052] acpiphp: Slot [29] registered [ 0.298079] acpiphp: Slot [30] registered [ 0.299091] acpiphp: Slot [31] registered [ 0.300076] PCI host bridge to bus 0000:00 [ 0.301015] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.302016] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.304019] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.306018] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.307018] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.309038] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.310239] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.313150] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.315257] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.324824] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.329500] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.331020] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.333079] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.335016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.337083] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.339971] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.342047] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.343879] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.350014] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.364015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.368011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.372812] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.380016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.399022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.426016] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.437054] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.445017] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.453016] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.488023] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.502694] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.516017] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.522028] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.551018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.561833] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.573016] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.585016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.609018] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.624703] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.633018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.640017] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.655019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.667000] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.679017] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.687016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.701017] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.716000] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.718460] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.721457] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.725426] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.726217] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.730146] iommu: Default domain type: Passthrough [ 0.732427] SCSI subsystem initialized [ 0.733186] ACPI: bus type USB registered [ 0.734089] usbcore: registered new interface driver usbfs [ 0.736062] usbcore: registered new interface driver hub [ 0.737056] usbcore: registered new device driver usb [ 0.738151] pps_core: LinuxPPS API ver. 1 registered [ 0.739011] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.742069] PTP clock support registered [ 0.744066] EDAC MC: Ver: 3.0.0 [ 0.745383] PCI: Using ACPI for IRQ routing [ 0.747982] NetLabel: Initializing [ 0.749016] NetLabel: domain hash size = 128 [ 0.751015] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.753085] NetLabel: unlabeled traffic allowed by default [ 0.756043] vgaarb: loaded [ 0.757276] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.759013] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.768228] clocksource: Switched to clocksource kvm-clock [ 0.886534] VFS: Disk quotas dquot_6.6.0 [ 0.888061] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.890749] *** VALIDATE ramfs *** [ 0.891853] *** VALIDATE hugetlbfs *** [ 0.893555] pnp: PnP ACPI init [ 0.895636] pnp: PnP ACPI: found 6 devices [ 0.912682] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.915312] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.916911] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.919177] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.921555] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.923241] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.925398] NET: Registered protocol family 2 [ 0.927853] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.933152] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.936970] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.942498] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.946369] TCP: Hash tables configured (established 65536 bind 65536) [ 0.949534] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.954350] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.957312] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.960329] NET: Registered protocol family 1 [ 0.962850] RPC: Registered named UNIX socket transport module. [ 0.965210] RPC: Registered udp transport module. [ 0.967106] RPC: Registered tcp transport module. [ 0.968938] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.971356] NET: Registered protocol family 44 [ 0.973257] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.975666] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.977943] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.980582] PCI: CLS 0 bytes, default 64 [ 0.982918] Unpacking initramfs... [ 4.216060] hrtimer: interrupt took 4867959 ns [ 5.001904] debug: unmapping init [mem 0xffff92e1fcc54000-0xffff92e1fffbffff] [ 5.022396] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 5.032021] software IO TLB: mapped [mem 0x00000000a3200000-0x00000000a7200000] (64MB) [ 5.043436] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 6.076476] Initialise system trusted keyrings [ 6.078371] Key type blacklist registered [ 6.080318] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 6.092561] zbud: loaded [ 6.095728] *** VALIDATE nfs *** [ 6.097041] *** VALIDATE nfs4 *** [ 6.098958] pstore: using deflate compression [ 6.103354] Platform Keyring initialized [ 6.556626] NET: Registered protocol family 38 [ 6.569657] Key type asymmetric registered [ 6.575609] Asymmetric key parser 'x509' registered [ 6.590835] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 6.628891] io scheduler mq-deadline registered [ 6.633661] io scheduler kyber registered [ 6.637087] io scheduler bfq registered [ 6.639955] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 6.649602] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 6.654540] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 6.662619] ACPI: Power Button [PWRF] [ 6.679459] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 6.699098] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 6.732063] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 6.747514] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 6.791391] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 6.850413] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 6.933633] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 6.951292] Non-volatile memory driver v1.3 [ 6.953577] Linux agpgart interface v0.103 [ 7.035454] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 7.038605] vda: detected capacity change from 0 to 74698752 [ 7.074726] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 7.087570] vdb: detected capacity change from 0 to 1073741824 [ 7.138259] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.145696] vdc: detected capacity change from 0 to 2621440000 [ 7.193305] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 7.208019] vdd: detected capacity change from 0 to 2621440000 [ 7.276996] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.281086] vde: detected capacity change from 0 to 4294967296 [ 7.337472] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 7.360433] vdf: detected capacity change from 0 to 4294967296 [ 7.382534] libphy: Fixed MDIO Bus: probed [ 7.398429] usbcore: registered new interface driver usbserial_generic [ 7.400674] usbserial: USB Serial support registered for generic [ 7.403474] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 7.406805] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 7.408829] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 7.411807] mousedev: PS/2 mouse device common for all mice [ 7.414487] rtc_cmos 00:05: RTC can wake from S4 [ 7.418022] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 7.418232] rtc_cmos 00:05: registered as rtc0 [ 7.425871] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 7.431111] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 7.431383] intel_pstate: CPU model not supported [ 7.438201] hid: raw HID events driver (C) Jiri Kosina [ 7.440139] usbcore: registered new interface driver usbhid [ 7.442286] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 7.442578] usbhid: USB HID core driver [ 7.443034] drop_monitor: Initializing network drop monitor service [ 7.443201] Initializing XFRM netlink socket [ 7.443626] NET: Registered protocol family 10 [ 7.460218] Segment Routing with IPv6 [ 7.461458] NET: Registered protocol family 17 [ 7.463933] mpls_gso: MPLS GSO support [ 7.470681] RAS: Correctable Errors collector initialized. [ 7.472700] AVX version of gcm_enc/dec engaged. [ 7.482709] AES CTR mode by8 optimization enabled [ 7.743230] sched_clock: Marking stable (7743194505, 0)->(8673889360, -930694855) [ 7.752474] registered taskstats version 1 [ 7.757181] Loading compiled-in X.509 certificates [ 7.765226] zswap: loaded using pool lzo/zbud [ 7.887502] Key type big_key registered [ 7.940834] Key type encrypted registered [ 7.943756] ima: No TPM chip found, activating TPM-bypass! [ 7.947615] ima: Allocated hash algorithm: sha1 [ 7.949115] ima: No architecture policies found [ 7.955592] evm: Initialising EVM extended attributes: [ 7.962163] evm: security.selinux [ 7.968067] evm: security.ima [ 7.969222] evm: security.capability [ 7.971271] evm: HMAC attrs: 0x1 [ 7.977852] rtc_cmos 00:05: setting system clock to 2026-08-14 21:44:45 UTC (1786743885) [ 8.003204] debug: unmapping init [mem 0xffffffffb9403000-0xffffffffb95fffff] [ 8.019523] debug: unmapping init [mem 0xffffffffb8182000-0xffffffffb8458fff] [ 8.032298] Write protecting the kernel read-only data: 28672k [ 8.067848] debug: unmapping init [mem 0xffffffffb6803000-0xffffffffb69fffff] [ 8.077511] debug: unmapping init [mem 0xffffffffb7114000-0xffffffffb71fffff] [ 8.343361] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 8.382382] systemd[1]: Detected virtualization kvm. [ 8.386517] systemd[1]: Detected architecture x86-64. [ 8.391478] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 8.463839] systemd[1]: No hostname configured. [ 8.476290] systemd[1]: Set hostname to . [ 8.483247] random: systemd: uninitialized urandom read (16 bytes read) [ 8.492991] systemd[1]: Initializing machine ID from random generator. [ 8.754166] random: ln: uninitialized urandom read (6 bytes read) [ 9.235097] random: systemd: uninitialized urandom read (16 bytes read) [ 9.252610] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 9.276368] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ 9.297533] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Listening on Journal Socket (/dev/log). Starting Journal Service... [ OK ] Listening on udev Control Socket. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Memstrack Anylazing Service. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Create Volatile Files and Directories. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 11.723699] device-mapper: uevent: version 1.0.3 [ 11.726478] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. Starting dracut initqueue hook... [ OK ] Reached target System Initialization. [ 13.699914] virtio_net virtio0 ens2: renamed from eth0 [ OK ] Reached target Basic System. [ 13.726682] random: fast init done [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ 13.923481] scsi host0: ata_piix [ 13.972504] scsi host1: ata_piix [ 13.974243] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 13.981911] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 18.495524] random: crng init done [ 18.497508] random: 7 urandom warning(s) missed due to ratelimiting [ 22.084248] dracut-initqueue[590]: RTNETLINK answers: File exists Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 24.040949] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 27.718842] printk: systemd: 26 output lines suppressed due to ratelimiting [ 28.728441] SELinux: Disabled at runtime. [ 28.942576] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 28.971133] systemd[1]: Detected virtualization kvm. [ 28.978189] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 31.262849] systemd[1]: initrd-switch-root.service: Succeeded. [ 31.277843] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 31.301506] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 31.320840] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 31.331755] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 31.385458] systemd[1]: Starting Journal Service... Starting Journal Service... [ 31.415183] systemd[1]: Listening on udev Control Socket. [ OK ] Listening on udev Control Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting POSIX Message Queue File System... [ 31.554240] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Created slice system-getty.slice. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. Starting Create list of required st…ce nodes for the current kernel... Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Listening on Process Core Dump Socket. [ OK ] Reached target RPC Port Mapper. Mounting Kernel Debug File System... [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Starting udev Coldplug all Devices... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd Root File System. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Remount Root and Kernel File Systems... [ OK ] Started Journal Service. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 33.008428] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 34.089645] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 34.382703] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 35.238313] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 35.350302] EDAC sbridge: Ver: 1.1.2 [* ] A start job is running for Configur…-only root support (8s / no limit) [** ] A start job is running for Configur…-only root support (8s / no limit)[ 40.109441] Key type dns_resolver registered [*** ] A start job is running for Configur…-only root support (9s / no limit) [ *** ] A start job is running for Configur…-only root support (9s / no limit)[ 40.754663] NFS: Registering the id_resolver key type [ 40.757312] Key type id_resolver registered [ 40.763433] Key type id_legacy registered [ *** ] A start job is running for Configur…only root support (10s / no limit) [ ***] A start job is running for Configur…only root support (10s / no limit) [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Login Service... Starting Network Manager... [ OK ] Started irqbalance daemon. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started OpenSSH server daemon. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting System Logging Service... Starting Crash recovery kernel arming... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg124-server login: [ 122.266701] libcfs: loading out-of-tree module taints kernel. [ 122.352932] Key type ._llcrypt registered [ 122.354338] Key type .llcrypt registered [ 122.530261] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 143.538552] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 145.389682] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 145.422172] alg: No test for adler32 (adler32-zlib) [ 146.872274] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 147.844855] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 149.752370] Key type lgssc registered [ 151.421426] Lustre: Echo OBD driver; http://www.lustre.org/ [ 171.862710] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 216.128413] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 235.279167] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 235.306148] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 236.688878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 236.711225] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 236.798474] Lustre: lustre-MDT0000: new disk, initializing [ 236.929646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 236.953532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 241.299470] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 254.553265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 254.640805] Lustre: 6531:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 254.665326] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 254.669581] Lustre: Skipped 1 previous similar message [ 254.753510] Lustre: lustre-MDT0001: new disk, initializing [ 254.819785] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 254.871819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 254.907079] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 259.648108] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 264.090551] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 274.108926] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 274.446021] Lustre: lustre-OST0000: new disk, initializing [ 274.450548] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 274.459429] Lustre: 8467:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 274.549386] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 275.159855] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 275.178540] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 275.242729] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 281.258200] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 295.756423] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 295.950447] Lustre: lustre-OST0001: new disk, initializing [ 295.957647] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 295.964422] Lustre: 9541:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 296.053378] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 301.705582] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 301.718266] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 301.864520] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 303.125719] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 315.074801] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 321.079148] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 327.164457] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing check_logdir /tmp/testlogs/ [ 332.608174] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing yml_node [ 337.834452] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 341.663269] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 344.828649] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 346.534770] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Aug 14 17:50:21 EDT 2026 [ 365.495649] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 367.164833] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 382.947073] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 382.959821] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 382.974151] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 383.972259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 383.973140] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 387.630401] Lustre: server umount lustre-MDT0000 complete [ 391.659203] LustreError: 6522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744269 with bad export cookie 1236775289535023466 [ 391.665213] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 391.672868] LustreError: 6522:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 394.209110] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 394.213919] LustreError: 6536:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 394.214109] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 394.214113] Lustre: Skipped 3 previous similar messages [ 394.225109] Lustre: Skipped 3 previous similar messages [ 394.268893] LustreError: 6536:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 396.257227] LustreError: 6538:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 396.260172] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 396.286120] Lustre: Skipped 2 previous similar messages [ 398.331819] Lustre: server umount lustre-MDT0001 complete [ 402.814561] Lustre: server umount lustre-OST0000 complete [ 406.366701] Lustre: server umount lustre-OST0001 complete [ 413.753904] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 423.780342] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 460.752168] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 461.069334] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 461.110863] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 461.268718] Lustre: lustre-MDT0000: new disk, initializing [ 461.415982] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 461.456666] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 465.200892] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 475.973465] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 476.092287] Lustre: 15093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 476.156880] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 476.179248] Lustre: Skipped 1 previous similar message [ 476.275917] Lustre: lustre-MDT0001: new disk, initializing [ 476.409472] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 476.444634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 476.455432] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 480.879087] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 490.488846] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 495.637757] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 502.711837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 502.954289] Lustre: lustre-OST0000: new disk, initializing [ 502.957590] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 502.962613] Lustre: 17063:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 503.048402] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 504.613778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 504.628801] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 504.773891] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 510.740902] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 526.009693] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 533.822636] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 534.293616] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 540.772398] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 541.122523] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 545.767954] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 545.789853] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 545.792680] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 545.812396] Lustre: Skipped 1 previous similar message [ 549.571395] Lustre: server umount lustre-OST0000 complete [ 553.442108] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 553.461980] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 553.484046] Lustre: Skipped 1 previous similar message [ 556.001282] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 556.021263] Lustre: Skipped 2 previous similar messages [ 559.721725] Lustre: server umount lustre-MDT0000 complete [ 561.122498] LustreError: 15101:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 561.144034] LustreError: 15101:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 563.450994] LustreError: 15087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744440 with bad export cookie 1236775289535024999 [ 563.459894] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 563.463050] LustreError: 15087:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 563.821451] Lustre: server umount lustre-MDT0001 complete [ 568.236620] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:54:02 (1786744442) [ 573.344150] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 573.959576] LustreError: 19327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 574.033532] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 577.637048] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 579.044190] LustreError: 19327:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 579.085318] LustreError: 19327:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 586.244584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 591.638945] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 599.390640] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 605.966408] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 615.042517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 621.997813] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 629.734315] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 637.063736] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 647.138078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 647.147748] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 647.154743] Lustre: Skipped 3 previous similar messages [ 647.183815] Lustre: Skipped 2 previous similar messages [ 651.881240] Lustre: server umount lustre-MDT0000 complete [ 652.260491] LustreError: 20580:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 652.305287] LustreError: 20580:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 657.038873] LustreError: 19307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744534 with bad export cookie 1236775289535025881 [ 657.053336] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 657.057483] LustreError: 19307:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 657.746722] Lustre: server umount lustre-MDT0001 complete [ 664.928573] Lustre: server umount lustre-OST0000 complete [ 690.339659] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 701.986435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 702.563330] LustreError: 23710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 702.591997] LustreError: 23710:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 702.658940] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 702.666626] Lustre: Skipped 2 previous similar messages [ 707.279833] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 718.209963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 723.135411] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 726.106340] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 732.286704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 739.857841] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 750.225454] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 750.470434] Lustre: lustre-OST0001: new disk, initializing [ 750.473939] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 750.480786] Lustre: 26026:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 752.503778] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 752.522992] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 752.629381] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 757.972703] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 769.203506] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 773.286933] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 788.450559] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 788.456993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 788.472048] Lustre: Skipped 3 previous similar messages [ 788.486677] Lustre: Skipped 3 previous similar messages [ 792.355868] Lustre: server umount lustre-MDT0000 complete [ 793.574905] LustreError: 24732:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 793.603465] LustreError: 24732:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 796.100242] LustreError: 23690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744673 with bad export cookie 1236775289535026728 [ 796.109736] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 796.115837] LustreError: 23690:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 796.384949] Lustre: server umount lustre-MDT0001 complete [ 811.471122] Lustre: server umount lustre-OST0000 complete [ 825.671689] Lustre: server umount lustre-OST0001 complete [ 842.375101] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 853.182838] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 853.530385] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 853.536127] Lustre: Skipped 3 previous similar messages [ 858.295887] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 858.600709] LustreError: 29048:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 858.619690] LustreError: 29048:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 868.335358] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 873.828080] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 877.130052] Lustre: 30195:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 885.405480] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 893.038267] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 902.900346] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 910.157496] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 917.514211] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 920.876405] Lustre: 32075:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 925.155532] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 925.161735] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 925.168517] Lustre: Skipped 1 previous similar message [ 925.174795] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 931.326917] Lustre: server umount lustre-MDT0000 complete [ 935.622311] LustreError: 29028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744813 with bad export cookie 1236775289535028023 [ 935.625528] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 935.640618] LustreError: 29028:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 935.935738] Lustre: server umount lustre-MDT0001 complete [ 949.916833] Lustre: server umount lustre-OST0000 complete [ 963.523472] Lustre: server umount lustre-OST0001 complete [ 972.394575] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 18:00:47 (1786744847) [ 1006.145321] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1019.543428] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1020.022859] LustreError: 34892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1020.049232] LustreError: 34892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 1027.102101] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1035.390958] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1041.139688] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1044.799161] Lustre: 36043:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1053.660182] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1063.577810] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1072.117941] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1081.970271] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1092.059470] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1096.245251] Lustre: 37924:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1102.305755] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1102.324130] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1102.342482] Lustre: Skipped 3 previous similar messages [ 1102.347981] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1102.354060] Lustre: Skipped 4 previous similar messages [ 1107.846864] Lustre: server umount lustre-MDT0000 complete [ 1111.533782] LustreError: 37990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786744989 with bad export cookie 1236775289535029129 [ 1111.539673] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1111.803271] Lustre: server umount lustre-MDT0001 complete [ 1124.720955] Lustre: server umount lustre-OST0000 complete [ 1139.251266] Lustre: server umount lustre-OST0001 complete [ 1147.664701] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 18:03:42 (1786745022) [ 1165.158907] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1176.905040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1177.277922] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1177.282086] Lustre: Skipped 7 previous similar messages [ 1181.174744] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1189.530900] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1193.263743] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1195.789389] Lustre: 41214:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1201.612444] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1208.719651] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1218.538195] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1225.175637] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1233.197424] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1238.122235] Lustre: 43091:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1239.567674] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1239.576846] Lustre: Skipped 1 previous similar message [ 1244.650055] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1244.717168] Lustre: Skipped 5 previous similar messages [ 1246.380358] Lustre: server umount lustre-MDT0000 complete [ 1251.614270] LustreError: 40048:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786745129 with bad export cookie 1236775289535030235 [ 1251.636536] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1258.382459] Lustre: server umount lustre-MDT0001 complete [ 1270.439402] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1278.748076] LustreError: 41578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1278.777180] LustreError: 41578:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 30 previous similar messages [ 1284.453142] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1287.153708] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1294.361722] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1299.919965] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1299.953150] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1299.960958] Lustre: Skipped 1 previous similar message [ 1308.683972] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1310.519465] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1316.344880] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1318.467664] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1325.595248] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1331.948982] Lustre: server umount lustre-MDT0000 complete [ 1335.707619] LustreError: 40047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786745213 with bad export cookie 1236775289535031341 [ 1335.724578] LustreError: 40047:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1335.743345] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1336.016542] Lustre: server umount lustre-MDT0001 complete [ 1345.447749] Lustre: server umount lustre-OST0000 complete [ 1348.861499] Lustre: server umount lustre-OST0001 complete [ 1355.860958] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 18:07:11 (1786745231) [ 1369.616828] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1378.874636] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1383.774915] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1392.258541] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1397.578286] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1400.627820] Lustre: 48850:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1406.751042] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1412.323555] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1421.796325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1429.342504] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1438.910474] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1446.317415] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1460.202892] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1460.210119] Lustre: Skipped 24 previous similar messages [ 1462.511099] Lustre: server umount lustre-MDT0000 complete [ 1465.870912] LustreError: 47677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786745343 with bad export cookie 1236775289535032160 [ 1465.883526] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1465.893253] LustreError: 47677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1466.238811] Lustre: server umount lustre-MDT0001 complete [ 1478.804386] Lustre: server umount lustre-OST0000 complete [ 1493.241480] Lustre: server umount lustre-OST0001 complete [ 1503.443541] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 18:09:38 (1786745378) [ 1505.762627] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1507.753968] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 18:09:42 (1786745382) [ 1518.309107] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 1526.767913] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1552.814517] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1553.043276] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1553.135215] Lustre: lustre-MDT0000: new disk, initializing [ 1553.269526] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1557.671131] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1567.079324] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1573.922157] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1574.173477] Lustre: lustre-OST0000: new disk, initializing [ 1574.177524] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1574.183142] Lustre: Skipped 1 previous similar message [ 1574.190260] Lustre: 54923:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1576.325752] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1576.334818] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1576.368071] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1580.193529] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1588.672988] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1590.815472] Lustre: DEBUG MARKER: create test files [ 1622.498315] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1622.517598] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1622.545575] Lustre: Skipped 12 previous similar messages [ 1626.431782] Lustre: server umount lustre-OST0000 complete [ 1630.932242] Lustre: server umount lustre-MDT0000 complete [ 1642.716053] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1647.157369] Key type lgssc unregistered [ 1647.526799] LNet: 56558:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1647.536333] LNetError: 56558:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1648.626960] LNet: Removed LNI 192.168.201.124@tcp [ 1649.957253] Key type .llcrypt unregistered [ 1649.958539] Key type ._llcrypt unregistered [ 1682.002247] Key type ._llcrypt registered [ 1682.004872] Key type .llcrypt registered [ 1682.160595] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1684.200734] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1684.275622] alg: No test for adler32 (adler32-zlib) [ 1685.277740] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 1685.472423] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1687.104242] Key type lgssc registered [ 1688.367834] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1698.055469] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1698.073835] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1699.572475] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1704.693655] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1712.093732] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1719.219994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1719.587603] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1724.747891] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1724.850862] Lustre: 59293:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1724.866922] Lustre: 59293:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1724.877032] Lustre: 59293:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1724.885284] Lustre: 59293:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1724.893251] Lustre: 59293:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1724.903868] Lustre: 59293:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1727.204383] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1735.532318] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1741.066475] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1745.750523] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1752.491684] Lustre: server umount lustre-OST0000 complete [ 1766.525256] Lustre: server umount lustre-MDT0000 complete [ 1776.565677] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1778.962342] Key type lgssc unregistered [ 1779.222870] LNet: 60909:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1779.229838] LNetError: 60909:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1779.239542] LNet: Removed LNI 192.168.201.124@tcp [ 1780.059309] Key type .llcrypt unregistered [ 1780.065431] Key type ._llcrypt unregistered [ 1808.973775] Key type ._llcrypt registered [ 1808.978146] Key type .llcrypt registered [ 1809.083726] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1810.417055] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1810.441184] alg: No test for adler32 (adler32-zlib) [ 1811.472345] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 1811.729667] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1813.425390] Key type lgssc registered [ 1814.451212] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1822.380482] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1822.423100] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1823.832871] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1828.921733] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1839.827353] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1848.463625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1849.069415] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1850.105395] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1850.239865] Lustre: 63746:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1850.246374] Lustre: 63746:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1850.250452] Lustre: 63746:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1850.260770] Lustre: 63746:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1850.269743] Lustre: 63746:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1850.274356] Lustre: 63746:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1855.992680] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1864.951163] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1870.629773] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1881.059530] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1881.066763] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1881.077241] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1883.664617] Lustre: server umount lustre-OST0000 complete [ 1887.445863] Lustre: server umount lustre-MDT0000 complete [ 1896.381851] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 1898.537736] Key type lgssc unregistered [ 1898.741892] LNet: 65306:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1898.749546] LNetError: 65306:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1898.771277] LNet: Removed LNI 192.168.201.124@tcp [ 1899.526460] Key type .llcrypt unregistered [ 1899.529888] Key type ._llcrypt unregistered [ 1915.189572] Key type ._llcrypt registered [ 1915.197224] Key type .llcrypt registered [ 1915.309721] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 1928.856541] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 1929.846450] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1930.027486] alg: No test for adler32 (adler32-zlib) [ 1931.087985] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 1931.327323] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 1932.984517] Key type lgssc registered [ 1933.842696] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1967.933689] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1967.954864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1969.332654] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1969.371141] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1969.552679] Lustre: lustre-MDT0000: new disk, initializing [ 1969.673841] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1969.714547] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1974.568899] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 1986.689595] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1986.817302] Lustre: 68458:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1986.873218] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1986.877331] Lustre: Skipped 1 previous similar message [ 1986.964936] Lustre: lustre-MDT0001: new disk, initializing [ 1987.148079] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1987.205274] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1987.219497] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1991.710551] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2002.300946] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2007.902655] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2014.517947] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2014.729554] Lustre: lustre-OST0000: new disk, initializing [ 2014.733593] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2014.739665] Lustre: 70429:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2014.830470] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2020.811912] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2020.954658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2020.970531] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2021.131354] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2030.024729] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2034.556169] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2034.743498] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2038.831774] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2039.122694] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2041.829733] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2041.833934] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2041.835027] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2041.841685] LustreError: Skipped 1 previous similar message [ 2041.867175] Lustre: Skipped 1 previous similar message [ 2046.949617] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2046.954500] Lustre: Skipped 2 previous similar messages [ 2052.071454] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2052.077517] Lustre: Skipped 1 previous similar message [ 2055.136183] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2055.303748] Lustre: server umount lustre-OST0000 complete [ 2062.306328] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2062.311375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2062.321902] Lustre: Skipped 1 previous similar message [ 2062.330017] Lustre: Skipped 1 previous similar message [ 2065.044380] Lustre: server umount lustre-MDT0000 complete [ 2067.757474] LustreError: 69254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786745945 with bad export cookie 13791459342390104258 [ 2067.760024] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2067.766629] LustreError: 69254:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2068.092551] Lustre: server umount lustre-MDT0001 complete [ 2076.495953] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 18:19:11 (1786745951) [ 2087.920318] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2125.938523] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 2136.456785] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2179.758089] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2180.022771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2180.067421] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2180.125496] Lustre: lustre-MDT0000: new disk, initializing [ 2180.178314] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2180.191926] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2184.450299] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2195.479481] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2195.657120] Lustre: 76448:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2195.781619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2195.786686] Lustre: Skipped 1 previous similar message [ 2195.894584] Lustre: lustre-MDT0001: new disk, initializing [ 2196.046331] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2196.086860] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2196.107749] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2201.913870] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2213.222275] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2220.931512] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2228.180639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2228.584837] Lustre: lustre-OST0000: new disk, initializing [ 2228.590191] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2228.606096] Lustre: 78419:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2228.790628] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2234.505728] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2234.515315] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2234.667103] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2235.957610] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2245.928445] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2250.833429] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2251.034614] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2256.294428] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2256.757667] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2260.463225] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2260.465820] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2260.482697] LustreError: Skipped 1 previous similar message [ 2260.486308] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2260.540673] Lustre: Skipped 1 previous similar message [ 2270.691903] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2270.700158] Lustre: Skipped 4 previous similar messages [ 2273.251237] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2273.696131] Lustre: server umount lustre-OST0000 complete [ 2280.929562] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2280.948838] Lustre: Skipped 1 previous similar message [ 2284.562613] Lustre: server umount lustre-MDT0000 complete [ 2286.063275] LustreError: 76460:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2286.087153] LustreError: 76460:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2288.423896] LustreError: 76439:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786746165 with bad export cookie 13791459342390105140 [ 2288.438137] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2288.931963] Lustre: server umount lustre-MDT0001 complete [ 2299.394521] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 18:22:53 (1786746173) [ 2305.624695] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2305.915715] LustreError: 80832:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2305.969264] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2309.814593] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2311.138049] LustreError: 80833:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2316.263824] LustreError: 80832:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2317.823886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2322.719643] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2330.258861] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2336.412767] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2343.501625] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2343.933358] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2343.942321] Lustre: Skipped 1 previous similar message [ 2351.752621] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2360.584200] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2364.162421] Lustre: 83503:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 411efeb2-657f-4481-904d-e4ef1926b718 at adminstrative request [ 2368.486677] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2368.505629] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2378.726295] Lustre: MGS: Client 411efeb2-657f-4481-904d-e4ef1926b718 (at 0@lo) reconnecting [ 2378.745953] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0xbf651b339d6797a6 to 0xbf651b339d679a46 [ 2378.761633] Lustre: MGC192.168.201.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2402.010478] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2402.187728] Lustre: lustre-OST0001: new disk, initializing [ 2402.191630] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2402.196974] Lustre: 83898:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2408.414646] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2411.583882] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2411.599481] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2411.706861] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2419.090475] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2422.247046] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2422.262667] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2422.269819] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2422.269828] Lustre: Skipped 2 previous similar messages [ 2428.472659] Lustre: server umount lustre-OST0001 complete [ 2432.485980] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2432.496901] LustreError: 84379:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2432.502673] Lustre: Skipped 2 previous similar messages [ 2432.528329] LustreError: 84379:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2442.726107] LustreError: 82652:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2442.751274] LustreError: 82652:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2446.306400] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2446.470464] Lustre: server umount lustre-OST0000 complete [ 2451.425585] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2451.431758] LustreError: Skipped 1 previous similar message [ 2451.434550] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2458.085292] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2458.091569] Lustre: Skipped 13 previous similar messages [ 2464.224365] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2464.643252] Lustre: server umount lustre-MDT0000 complete [ 2468.336282] LustreError: 80833:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2468.385461] LustreError: 80833:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2469.355255] LustreError: 80813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786746346 with bad export cookie 13791459342390106694 [ 2469.367320] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2469.379729] LustreError: 80813:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2469.911404] Lustre: server umount lustre-MDT0001 complete [ 2480.540746] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:25:54 (1786746354) [ 2482.110466] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2483.534323] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:25:58 (1786746358) [ 2485.153932] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2486.545288] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:26:01 (1786746361) [ 2487.780768] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2490.101465] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:26:04 (1786746364) [ 2491.666925] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2493.410278] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:26:08 (1786746368) [ 2494.975217] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2501.078458] loop: module loaded [ 2501.109215] loop0: detected capacity change from 0 to 4294967296 [ 2501.144637] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2502.586650] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2505.981849] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2509.700185] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2535.076092] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2535.130357] LustreError: 88024:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2535.147619] LustreError: 88024:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2535.159857] LustreError: 88024:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2535.167545] LustreError: 88024:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2555.678301] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 2564.660268] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 2602.377108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2602.593218] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2602.639630] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2602.729295] Lustre: lustre-MDT0000: new disk, initializing [ 2602.924955] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2602.931864] Lustre: Skipped 1 previous similar message [ 2602.958613] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2606.846882] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2617.978846] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2618.108884] Lustre: 90891:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2618.455111] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2623.416215] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2633.351985] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2640.286490] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2647.627735] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2648.169478] Lustre: 92858:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2653.247992] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2653.406806] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2654.826792] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2663.595821] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2668.731912] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2668.923523] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2673.643138] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2673.841313] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2679.272896] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2679.273121] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2679.274450] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2679.290719] Lustre: Skipped 2 previous similar messages [ 2679.352091] Lustre: Skipped 3 previous similar messages [ 2690.021991] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2690.185183] Lustre: server umount lustre-OST0000 complete [ 2695.137870] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2701.476742] Lustre: server umount lustre-MDT0000 complete [ 2704.867126] LustreError: 90902:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2704.894371] LustreError: 90902:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2705.433895] LustreError: 90882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786746582 with bad export cookie 13791459342390107310 [ 2705.442598] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2705.455574] LustreError: 90882:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2706.249172] Lustre: server umount lustre-MDT0001 complete [ 2716.104634] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:29:50 (1786746590) [ 2724.821532] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2729.377919] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2738.004462] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2738.413321] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2738.425736] Lustre: Skipped 3 previous similar messages [ 2743.522960] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2751.057223] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2757.285852] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2765.537124] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2771.611389] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2778.192287] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2782.649884] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2782.872036] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2787.337820] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2787.572511] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2794.415907] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2794.646819] Lustre: lustre-OST0001: new disk, initializing [ 2794.652175] Lustre: Skipped 2 previous similar messages [ 2794.659093] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2794.663595] Lustre: Skipped 3 previous similar messages [ 2794.672491] Lustre: 98646:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2801.188487] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2801.738032] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2801.746110] Lustre: Skipped 2 previous similar messages [ 2801.752559] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2801.868600] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2810.779586] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2815.375403] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2815.597718] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2820.411987] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2820.660668] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2839.736918] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2859.297886] Lustre: Failing over lustre-MDT0000 [ 2859.957635] Lustre: server umount lustre-MDT0000 complete [ 2861.536756] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2861.540969] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2861.552985] Lustre: Skipped 2 previous similar messages [ 2861.566872] LustreError: 95277:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2861.592913] LustreError: 95277:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2879.989683] Lustre: 66686:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786746741/real 1786746741] req@0000000089fb896b x1873538772651648/t0(0) o400->MGC192.168.201.124@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786746757 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2880.017092] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2884.525520] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2884.530931] LDISKFS-fs (dm-0): recovery complete [ 2884.548986] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2890.240866] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0xbf651b339d6864ba [ 2890.257068] Lustre: MGC192.168.201.124@tcp: Connection restored to 0@lo (at 0@lo) [ 2890.555393] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2890.580835] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2890.582295] Lustre: 97319:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2894.037615] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 2895.858136] Lustre: 95277:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2895.869994] Lustre: 95277:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2895.889418] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 2895.992759] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2896.320335] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2896.652835] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2896.660986] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2897.000139] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2897.008577] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2897.022387] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2897.039437] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2897.681306] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2897.693477] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2898.024394] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2898.035243] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2899.315434] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2899.331299] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 6 previous similar messages [ 2899.974922] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2899.995294] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2900.321768] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2900.329828] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 6 previous similar messages [ 2903.490373] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2903.503591] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2904.154030] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2904.164551] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2904.480514] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2904.485124] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2911.762820] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2911.768686] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 25 previous similar messages [ 2912.424887] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2912.434043] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2912.753571] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2912.759566] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 25 previous similar messages [ 2928.018646] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2928.034587] Lustre: 101494:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 2928.667901] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2928.676267] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2929.000162] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2929.008517] LustreError: 101494:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 2950.304234] Lustre: 101494:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2950.319328] Lustre: 101494:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2950.333466] Lustre: 101494:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client f799b0f8-dee4-41b5-a154-9f1c7fa1f443@192.168.201.24@tcp [ 2950.358778] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2950.365603] Lustre: 101494:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@0000000023a0e7bf x1873538757755648/t0(8589934767) o101->f799b0f8-dee4-41b5-a154-9f1c7fa1f443@192.168.201.24@tcp:78/0 lens 592/0 e 0 to 0 dl 1786746838 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2950.407729] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2950.415848] Lustre: lustre-MDT0000: Denying connection for new client f799b0f8-dee4-41b5-a154-9f1c7fa1f443 (at 192.168.201.24@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2950.435232] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2950.488548] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2950.493067] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2950.496726] Lustre: Skipped 2 previous similar messages [ 2950.539546] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:44 to 0x2c0000401:545) [ 2950.546274] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2966.497475] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2966.508296] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2966.519718] Lustre: Skipped 7 previous similar messages [ 2980.320136] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2980.528835] Lustre: server umount lustre-OST0000 complete [ 2993.123787] LustreError: 97096:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2993.141660] LustreError: 97096:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 45 previous similar messages [ 2998.240636] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2998.508456] Lustre: server umount lustre-OST0001 complete [ 3003.366483] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3003.379835] Lustre: Skipped 7 previous similar messages [ 3008.817711] Lustre: server umount lustre-MDT0000 complete [ 3013.116210] LustreError: 95259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786746890 with bad export cookie 13791459342390158522 [ 3013.121022] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3013.138083] LustreError: 95259:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3013.584606] Lustre: server umount lustre-MDT0001 complete [ 3024.281909] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:34:58 (1786746898) [ 3031.681301] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3031.687195] Lustre: *** cfs_fail_loc=197, val=0*** [ 3047.264798] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3052.385451] LustreError: 103473:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.124@tcp: failed processing log, type 4: rc = -110 [ 3057.608827] Lustre: *** cfs_fail_loc=197, val=0*** [ 3057.611941] Lustre: Skipped 65 previous similar messages [ 3078.240459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3078.254165] Lustre: Skipped 3 previous similar messages [ 3085.460160] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3092.538336] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3094.308297] Lustre: server umount lustre-OST0000 complete [ 3104.847037] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:36:19 (1786746979) [ 3125.454908] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3130.642174] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3140.177604] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3143.875404] LustreError: 105124:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3143.898415] LustreError: 105124:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3143.914977] LustreError: 105124:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 18, retries 0, failed: rc = -5 [ 3144.203773] Lustre: server umount lustre-MDT0000 complete [ 3182.879328] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3193.750900] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3231.639946] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3231.900988] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3231.957294] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3232.131903] Lustre: lustre-MDT0000: new disk, initializing [ 3232.274096] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3236.998696] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3247.890553] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3248.089510] Lustre: 109300:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3248.531052] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3254.172648] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3263.869142] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3270.066374] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3276.895253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3277.295366] Lustre: 111270:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3284.585123] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3287.277042] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3295.249749] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3300.435838] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3300.706865] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3304.882563] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3305.111292] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3308.003871] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3308.005732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3308.013058] LustreError: Skipped 3 previous similar messages [ 3308.013714] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3308.013719] Lustre: Skipped 18 previous similar messages [ 3308.051290] Lustre: Skipped 2 previous similar messages [ 3313.907408] Lustre: server umount lustre-OST0000 complete [ 3325.266915] Lustre: server umount lustre-MDT0000 complete [ 3326.442599] LustreError: 110195:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3326.453673] LustreError: 110195:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 3329.408572] LustreError: 109293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786747206 with bad export cookie 13791459342390169078 [ 3329.414932] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3329.420099] LustreError: 109293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3329.680078] Lustre: server umount lustre-MDT0001 complete [ 3337.948492] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:40:12 (1786747212) [ 3347.455029] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 3350.416958] Key type lgssc unregistered [ 3350.738572] LNet: 113762:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3350.748293] LNetError: 113762:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3350.772134] LNet: Removed LNI 192.168.201.124@tcp [ 3352.052431] Key type .llcrypt unregistered [ 3352.058288] Key type ._llcrypt unregistered [ 3372.309546] Key type ._llcrypt registered [ 3372.313167] Key type .llcrypt registered [ 3372.514738] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3388.674409] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3389.935760] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3390.005282] alg: No test for adler32 (adler32-zlib) [ 3391.014979] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 3391.222041] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 3392.864187] Key type lgssc registered [ 3394.116316] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3433.273362] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3433.321146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3434.608384] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3434.652872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3434.822360] Lustre: lustre-MDT0000: new disk, initializing [ 3434.936176] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3434.957822] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3439.370814] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3449.298325] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3449.520538] Lustre: lustre-OST0000: new disk, initializing [ 3449.523737] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3449.527591] Lustre: Skipped 1 previous similar message [ 3449.532525] Lustre: 117961:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3449.592368] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3452.464610] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3452.480236] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3452.511265] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3456.277675] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3467.103682] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3467.259869] Lustre: lustre-OST0001: new disk, initializing [ 3467.266829] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3467.272638] Lustre: 118824:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3467.391375] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3473.654620] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3476.620185] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3476.629801] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3476.686016] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3486.690490] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3486.692276] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3486.700120] Lustre: Skipped 1 previous similar message [ 3486.715084] Lustre: Skipped 1 previous similar message [ 3489.887922] Lustre: server umount lustre-MDT0000 complete [ 3497.203574] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3501.169026] LustreError: 117136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786747378 with bad export cookie 1890333909014602494 [ 3501.185112] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3501.262617] Lustre: server umount lustre-OST0000 complete [ 3504.176251] Lustre: server umount lustre-OST0001 complete [ 3513.020433] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 18:43:08 (1786747388) [ 3534.442986] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 18:43:29 (1786747409) [ 3544.536136] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3568.955036] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:44:03 (1786747443) [ 3608.051617] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 3617.379661] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3652.631784] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3652.821107] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3652.871753] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3652.930428] Lustre: lustre-MDT0000: new disk, initializing [ 3653.050627] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3653.083774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3658.739263] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3669.433837] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3669.511489] Lustre: 126592:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3669.545553] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3669.548878] Lustre: Skipped 1 previous similar message [ 3669.598794] Lustre: lustre-MDT0001: new disk, initializing [ 3669.668262] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3669.727723] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3669.742373] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3673.501824] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3684.414377] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3691.197737] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3697.327107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3697.690972] Lustre: lustre-OST0000: new disk, initializing [ 3697.697777] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3697.707251] Lustre: 128558:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3697.850445] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3704.672616] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3705.936699] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3705.949204] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3706.078385] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3714.824936] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3719.983313] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3720.300049] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3726.261358] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3726.559874] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3731.944262] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3731.946992] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3731.951952] LustreError: Skipped 1 previous similar message [ 3731.954180] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3731.971697] Lustre: Skipped 1 previous similar message [ 3731.990150] Lustre: Skipped 1 previous similar message [ 3737.058734] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3737.086610] Lustre: Skipped 1 previous similar message [ 3742.181612] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3743.200168] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3743.374380] Lustre: server umount lustre-OST0000 complete [ 3746.788835] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3746.802551] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3746.810664] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3746.815454] Lustre: Skipped 1 previous similar message [ 3753.150524] Lustre: server umount lustre-MDT0000 complete [ 3756.817950] LustreError: 126582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786747634 with bad export cookie 1890333909014603719 [ 3756.827616] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3756.838084] LustreError: 126582:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 3757.353841] Lustre: server umount lustre-MDT0001 complete [ 3765.751215] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:47:20 (1786747640) [ 3771.939338] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3772.450111] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3777.116570] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3777.507206] LustreError: 130973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3777.532821] LustreError: 130973:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3781.614472] LustreError: 130974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3785.608597] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3790.820897] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3799.209317] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3805.812796] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3815.232333] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3815.539269] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3815.543932] Lustre: Skipped 1 previous similar message [ 3821.949629] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3829.762572] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3837.469152] Lustre: *** cfs_fail_loc=159, val=0*** [ 3853.708424] Lustre: lustre-MDT0000: Client 3015040b-f490-4657-bcf2-c72261be00fb (at 192.168.201.24@tcp) reconnecting [ 3855.189177] Lustre: *** cfs_fail_loc=159, val=0*** [ 3855.199781] Lustre: Skipped 5 previous similar messages [ 3871.662911] Lustre: lustre-MDT0000: Client 3015040b-f490-4657-bcf2-c72261be00fb (at 192.168.201.24@tcp) reconnecting [ 3877.348294] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3877.369389] Lustre: Skipped 1 previous similar message [ 3877.378973] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3877.389027] Lustre: Skipped 5 previous similar messages [ 3891.168316] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3891.291554] Lustre: server umount lustre-OST0000 complete [ 3897.831502] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3897.844103] Lustre: Skipped 1 previous similar message [ 3897.874476] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3897.886829] Lustre: Skipped 5 previous similar messages [ 3901.529642] Lustre: server umount lustre-MDT0000 complete [ 3902.960402] LustreError: 133691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3902.984546] LustreError: 133691:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 3905.086455] LustreError: 132816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786747782 with bad export cookie 1890333909014604601 [ 3905.093408] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3905.098194] LustreError: 132816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3905.478518] Lustre: server umount lustre-MDT0001 complete [ 3917.127811] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 3919.523288] Key type lgssc unregistered [ 3919.823658] LNet: 134750:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3919.827599] LNetError: 134750:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3920.872920] LNet: Removed LNI 192.168.201.124@tcp [ 3921.804242] Key type .llcrypt unregistered [ 3921.806692] Key type ._llcrypt unregistered [ 3945.218228] Key type ._llcrypt registered [ 3945.220708] Key type .llcrypt registered [ 3945.314244] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 3946.527536] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3946.618599] alg: No test for adler32 (adler32-zlib) [ 3947.666303] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 3947.954257] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 3949.601081] Key type lgssc registered [ 3950.753166] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3958.883399] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3958.907211] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3960.473446] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3964.913729] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3970.027379] LustreError: 136167:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3970.061552] LustreError: 136167:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3973.258125] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3973.608570] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3977.831135] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 3985.281324] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3991.057846] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3997.492270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3997.896381] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4003.533884] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4005.053722] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4010.079151] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4018.957450] Lustre: *** cfs_fail_loc=159, val=0*** [ 4035.440092] Lustre: lustre-MDT0000: Client 8e720935-b6b8-4f2d-a740-91ffeb3707cf (at 192.168.201.24@tcp) reconnecting [ 4037.387686] Lustre: *** cfs_fail_loc=159, val=0*** [ 4052.911678] Lustre: lustre-MDT0001: Client 8e720935-b6b8-4f2d-a740-91ffeb3707cf (at 192.168.201.24@tcp) reconnecting [ 4054.739587] Lustre: *** cfs_fail_loc=159, val=0*** [ 4054.748786] Lustre: Skipped 3 previous similar messages [ 4071.307248] Lustre: lustre-MDT0001: Client 8e720935-b6b8-4f2d-a740-91ffeb3707cf (at 192.168.201.24@tcp) reconnecting [ 4076.861292] Lustre: 139203:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4079.780509] Lustre: *** cfs_fail_loc=159, val=0*** [ 4079.782577] Lustre: Skipped 4 previous similar messages [ 4096.496974] Lustre: lustre-MDT0000: Client 0266b3b5-a863-4c32-83fd-18a87811731b (at 192.168.201.24@tcp) reconnecting [ 4098.601118] Lustre: *** cfs_fail_loc=159, val=0*** [ 4098.608135] Lustre: Skipped 14 previous similar messages [ 4113.821132] Lustre: lustre-MDT0000: Client 0266b3b5-a863-4c32-83fd-18a87811731b (at 192.168.201.24@tcp) reconnecting [ 4116.347331] Lustre: 139498:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4122.084586] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4122.092025] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4122.108264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4123.105453] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4123.113402] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4123.117760] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4123.130430] Lustre: Skipped 1 previous similar message [ 4127.694312] Lustre: server umount lustre-OST0000 complete [ 4132.321499] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4132.334450] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4132.351168] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4134.338351] Lustre: server umount lustre-MDT0000 complete [ 4138.467457] LustreError: 136894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4138.492975] LustreError: 136894:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4139.707662] LustreError: 136901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748017 with bad export cookie 2423887842586752256 [ 4139.712876] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4139.725661] LustreError: 136901:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4140.235674] Lustre: server umount lustre-MDT0001 complete [ 4151.896943] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4154.629634] Key type lgssc unregistered [ 4154.908807] LNet: 140553:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4154.915501] LNetError: 140553:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4154.930315] LNet: Removed LNI 192.168.201.124@tcp [ 4155.670373] Key type .llcrypt unregistered [ 4155.673955] Key type ._llcrypt unregistered [ 4180.670696] Key type ._llcrypt registered [ 4180.672541] Key type .llcrypt registered [ 4180.766544] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4181.848615] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4181.865261] alg: No test for adler32 (adler32-zlib) [ 4182.902645] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 4183.115283] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4184.768266] Key type lgssc registered [ 4185.921048] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4193.805087] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4193.833801] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4195.363465] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4200.094761] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4204.524794] LustreError: 141967:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4204.545189] LustreError: 141967:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4207.405121] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4207.662473] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4212.011683] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4219.219348] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4224.538888] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4230.993041] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4231.484609] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4232.508388] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4237.684374] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4239.702509] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4244.860479] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4251.721567] Lustre: 144550:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4254.887981] Lustre: 144598:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4256.042452] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4257.011584] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4261.945772] Lustre: 144744:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4262.939987] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4264.030224] Lustre: 144842:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4268.520298] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4268.524986] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4268.551373] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4270.561104] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4270.565547] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4270.570113] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4270.590348] Lustre: Skipped 1 previous similar message [ 4275.687145] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4275.700280] Lustre: Skipped 1 previous similar message [ 4280.809642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4280.824415] Lustre: Skipped 1 previous similar message [ 4281.826717] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4281.954686] Lustre: server umount lustre-OST0000 complete [ 4285.921626] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4285.924693] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4285.936720] Lustre: Skipped 1 previous similar message [ 4285.943295] Lustre: Skipped 1 previous similar message [ 4291.642313] Lustre: server umount lustre-MDT0000 complete [ 4295.436728] LustreError: 141947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748172 with bad export cookie 5024285842403205628 [ 4295.440841] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4295.454920] LustreError: 141947:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4295.840283] Lustre: server umount lustre-MDT0001 complete [ 4305.970915] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4308.544431] Key type lgssc unregistered [ 4308.809464] LNet: 145907:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4308.818947] LNetError: 145907:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4308.838795] LNet: Removed LNI 192.168.201.124@tcp [ 4309.624889] Key type .llcrypt unregistered [ 4309.626306] Key type ._llcrypt unregistered [ 4335.856993] Key type ._llcrypt registered [ 4335.859146] Key type .llcrypt registered [ 4335.974387] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4336.802231] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4336.939039] alg: No test for adler32 (adler32-zlib) [ 4337.934822] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 4338.166992] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4339.832576] Key type lgssc registered [ 4341.092845] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4349.727370] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4349.775355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4351.355235] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4356.164553] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4360.676353] LustreError: 147320:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4360.685732] LustreError: 147320:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4364.295374] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4364.760434] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4369.621332] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4377.924189] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4384.271742] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4391.378659] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4391.860491] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4395.988492] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4399.679598] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4410.328728] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4411.049989] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4417.269933] Lustre: *** cfs_fail_loc=159, val=0*** [ 4433.821851] Lustre: lustre-MDT0000: Client b6c0d6e5-ca98-4094-887c-8780544140c2 (at 192.168.201.24@tcp) reconnecting [ 4451.305911] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4451.308091] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4451.320314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4451.333187] Lustre: Skipped 1 previous similar message [ 4456.419538] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4456.424734] Lustre: Skipped 1 previous similar message [ 4456.682346] Lustre: server umount lustre-OST0000 complete [ 4461.540980] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4461.557312] Lustre: Skipped 1 previous similar message [ 4461.566353] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4461.572561] Lustre: Skipped 1 previous similar message [ 4466.668915] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4466.674445] Lustre: Skipped 1 previous similar message [ 4467.107803] Lustre: server umount lustre-MDT0000 complete [ 4471.165420] LustreError: 147301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748348 with bad export cookie 13329016043157557266 [ 4471.170817] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4471.175238] LustreError: 147301:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4471.536901] Lustre: server umount lustre-MDT0001 complete [ 4481.689574] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4484.234325] Key type lgssc unregistered [ 4484.522831] LNet: 150992:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4484.536636] LNetError: 150992:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4485.609821] LNet: Removed LNI 192.168.201.124@tcp [ 4486.395880] Key type .llcrypt unregistered [ 4486.398427] Key type ._llcrypt unregistered [ 4508.864850] Key type ._llcrypt registered [ 4508.870322] Key type .llcrypt registered [ 4509.076992] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4510.585153] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4510.675787] alg: No test for adler32 (adler32-zlib) [ 4511.800202] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 4512.165029] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4513.896313] Key type lgssc registered [ 4515.802392] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4524.172249] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4524.201297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4525.762960] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4530.169916] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4536.291701] LustreError: 152409:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4536.328740] LustreError: 152409:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4539.524829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4540.247136] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4544.697388] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4552.343805] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4559.176708] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4566.630024] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4566.967918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4571.062762] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4573.841662] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4582.094303] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4585.419111] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4605.960577] Lustre: lustre-OST0000: evicting 0302edaa-7bf0-42fd-9f0f-7a5ad283623f (at 192.168.201.24@tcp) by administrative request [ 4605.966172] Lustre: Skipped 1 previous similar message [ 4636.647797] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4636.649765] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4636.660764] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4636.705440] Lustre: Skipped 1 previous similar message [ 4640.262453] Lustre: server umount lustre-OST0000 complete [ 4646.882041] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4646.896183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4646.903292] Lustre: Skipped 2 previous similar messages [ 4649.623643] Lustre: server umount lustre-MDT0000 complete [ 4652.006325] LustreError: 152410:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4652.024437] LustreError: 152410:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4653.868969] LustreError: 152391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748531 with bad export cookie 9169509283435506119 [ 4653.873986] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4653.875751] LustreError: 152391:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4654.059615] Lustre: server umount lustre-MDT0001 complete [ 4665.532724] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4668.515338] Key type lgssc unregistered [ 4668.806082] LNet: 156492:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4668.821680] LNetError: 156492:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4668.843270] LNet: Removed LNI 192.168.201.124@tcp [ 4669.671775] Key type .llcrypt unregistered [ 4669.674860] Key type ._llcrypt unregistered [ 4735.580206] Key type ._llcrypt registered [ 4735.589334] Key type .llcrypt registered [ 4735.697202] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4737.094825] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4737.146612] alg: No test for adler32 (adler32-zlib) [ 4738.266776] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 4738.609488] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4740.339248] Key type lgssc registered [ 4741.188941] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4749.452874] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4749.469327] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4751.099360] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4756.554822] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4760.548734] LustreError: 158979:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4760.579640] LustreError: 158979:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4765.593600] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4765.674675] LustreError: 158980:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4766.048072] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4772.593796] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4780.755056] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4786.833345] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4792.632942] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4793.094589] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4796.228049] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4798.340787] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4804.996161] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4810.553987] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4820.965724] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4820.972375] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4820.994933] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4821.007665] Lustre: Skipped 1 previous similar message [ 4823.288731] Lustre: server umount lustre-OST0000 complete [ 4827.617185] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4827.623447] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4827.631503] Lustre: Skipped 1 previous similar message [ 4827.643274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4831.205394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4831.207685] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4833.217781] Lustre: server umount lustre-MDT0000 complete [ 4836.329302] LustreError: 160828:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4836.340645] LustreError: 160828:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4837.442271] LustreError: 158959:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748714 with bad export cookie 15298096541537490681 [ 4837.461314] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4838.142241] Lustre: server umount lustre-MDT0001 complete [ 4848.380391] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 4851.188495] Key type lgssc unregistered [ 4851.525995] LNet: 162545:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4851.532360] LNetError: 162545:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4851.545660] LNet: Removed LNI 192.168.201.124@tcp [ 4852.525823] Key type .llcrypt unregistered [ 4852.527611] Key type ._llcrypt unregistered [ 4880.150886] Key type ._llcrypt registered [ 4880.152291] Key type .llcrypt registered [ 4880.208835] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4881.189340] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4881.280415] alg: No test for adler32 (adler32-zlib) [ 4882.357830] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 4882.607500] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 4884.344355] Key type lgssc registered [ 4885.818493] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4900.046711] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 19:06:15 (1786748775) [ 4915.670707] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 4926.413895] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 4961.972261] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4961.999964] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4963.306293] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4963.345722] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4963.500625] Lustre: lustre-MDT0000: new disk, initializing [ 4963.652958] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4963.692789] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4967.949389] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4978.123474] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4978.254377] Lustre: 166818:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4978.299834] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4978.303991] Lustre: Skipped 1 previous similar message [ 4978.357405] Lustre: lustre-MDT0001: new disk, initializing [ 4978.436312] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4978.481845] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4978.494819] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4982.948705] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 4992.723930] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4998.994686] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5006.877091] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5007.088097] Lustre: lustre-OST0000: new disk, initializing [ 5007.091803] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5007.096864] Lustre: 168785:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5007.263354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5014.641075] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5014.651758] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5014.711870] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5014.739916] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5024.585297] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5035.494969] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5035.495432] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5035.496332] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5035.501701] Lustre: Skipped 1 previous similar message [ 5035.513110] LustreError: Skipped 1 previous similar message [ 5040.611712] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5040.620540] Lustre: Skipped 1 previous similar message [ 5045.734475] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5045.741109] Lustre: Skipped 2 previous similar messages [ 5047.264632] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5047.490876] Lustre: server umount lustre-OST0000 complete [ 5050.861277] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5050.877094] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5055.457612] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5055.468600] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5055.483257] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5056.917470] Lustre: server umount lustre-MDT0000 complete [ 5060.214562] LustreError: 166809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786748937 with bad export cookie 13628687121430339655 [ 5060.219362] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5060.225081] LustreError: 166809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5060.536365] Lustre: server umount lustre-MDT0001 complete [ 5070.824242] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 5073.448533] Key type lgssc unregistered [ 5073.755586] LNet: 170693:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5073.766486] LNetError: 170693:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5073.782324] LNet: Removed LNI 192.168.201.124@tcp [ 5074.607867] Key type .llcrypt unregistered [ 5074.610540] Key type ._llcrypt unregistered [ 5096.993336] Key type ._llcrypt registered [ 5096.996182] Key type .llcrypt registered [ 5097.125339] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 5098.264915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5098.503272] alg: No test for adler32 (adler32-zlib) [ 5099.665949] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 5099.917294] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 5101.649542] Key type lgssc registered [ 5102.712820] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5111.715377] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5111.772387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5113.411504] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5117.399735] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5122.552293] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5122.581491] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5127.661501] LustreError: 172109:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5128.591713] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5129.149561] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5133.633235] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5141.960697] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5147.736762] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5154.281983] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5154.599587] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5157.694360] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5161.209389] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 5167.877760] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5177.709763] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5177.726086] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5177.740839] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5177.767800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.790965] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5179.105267] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5179.135237] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5179.188394] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5179.210830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.224284] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5180.153740] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5180.170539] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5180.188286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.082095] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5181.086706] Lustre: Skipped 1 previous similar message [ 5181.089094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.097480] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5181.103282] Lustre: Skipped 1 previous similar message [ 5182.042610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.882441] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5182.884716] Lustre: Skipped 2 previous similar messages [ 5182.922467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5182.939517] Lustre: Skipped 2 previous similar messages [ 5182.947348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.135846] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5184.144921] Lustre: Skipped 2 previous similar messages [ 5184.148730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.160921] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5184.167816] Lustre: Skipped 2 previous similar messages [ 5185.164362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.600776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.037482] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5188.050596] Lustre: Skipped 3 previous similar messages [ 5188.106323] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5188.127607] Lustre: Skipped 3 previous similar messages [ 5188.132946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.522903] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5189.527726] Lustre: Skipped 3 previous similar messages [ 5189.532083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.544283] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5189.552264] Lustre: Skipped 3 previous similar messages [ 5190.601491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.045501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.208230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.701941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.869843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.739842] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5196.743523] Lustre: Skipped 6 previous similar messages [ 5196.759740] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5196.768672] Lustre: Skipped 6 previous similar messages [ 5196.773045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.807890] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5197.814159] Lustre: Skipped 6 previous similar messages [ 5197.817190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.860843] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5197.880234] Lustre: Skipped 6 previous similar messages [ 5199.426063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.681924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.075033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.211077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.013822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5204.842748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.808039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.761586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.893290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.883982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.728427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.401299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.283499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.070784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.035636] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5213.042520] Lustre: Skipped 15 previous similar messages [ 5213.065290] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5213.073216] Lustre: Skipped 15 previous similar messages [ 5213.078223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.849368] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5213.852598] Lustre: Skipped 15 previous similar messages [ 5213.863128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.871934] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5213.881828] Lustre: Skipped 15 previous similar messages [ 5214.813425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.799664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.764774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.566286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.811777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.268205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.222903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.039986] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5222.049918] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5222.056347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.040731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.204932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.976385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.007617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.708753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.584055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.726573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.970306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.793692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.932775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.847582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.785536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.747961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.862129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.005329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.743491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.876514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.838816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.641845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.584238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.390945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.602353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.167493] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5245.179872] Lustre: Skipped 31 previous similar messages [ 5245.197961] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5245.230199] Lustre: Skipped 31 previous similar messages [ 5245.234087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.285285] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5246.288300] Lustre: Skipped 31 previous similar messages [ 5246.293131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.300847] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5246.315751] Lustre: Skipped 31 previous similar messages [ 5247.482555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.735273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.934653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.069503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.868939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.874846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.671186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.121042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.849486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.529155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.904511] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5259.925232] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5259.963095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5261.624823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.913779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.725068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.656942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.675748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.507463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.359261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.186971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.992864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.021604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.125618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.259785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.095261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.030568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.048045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.126339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.944733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.880798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.606428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.504973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.233847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.996376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.866331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.681138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.523843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.383669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.286752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.181073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.878677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.742267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.558988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.358929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.205756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.977966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.774494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.499787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.163980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.909914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.730805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.769759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.601567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.380482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.140270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.781512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.519803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.157286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.967473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.753680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5302.632298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.359638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.140182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.930437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.902876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.949232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.787416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.559083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5309.485172] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5309.489926] Lustre: Skipped 68 previous similar messages [ 5309.507637] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5309.514718] Lustre: Skipped 68 previous similar messages [ 5309.519837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.557418] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5310.563961] Lustre: Skipped 68 previous similar messages [ 5310.567996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.581211] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5310.584076] Lustre: Skipped 68 previous similar messages [ 5311.493307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.397700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.160025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.987721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.708685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.451693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.014654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.890412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.599639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.592995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.421840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.324313] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5320.342398] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5320.348106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.326233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.461594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.423704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.406526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.531930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.571761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.444403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.327016] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.049053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.871498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.860340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.228979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.270430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.488283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.789776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.844533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.053939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.425442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.301751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.551703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.549754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.736594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.562569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.401924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.243360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.092096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.735719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.980546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.789458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.011061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.290579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.554346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.827798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.813360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.817079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.664126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.746149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.485714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.482110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.227961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.815306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.526320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.623244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.779227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.958268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.880059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.886569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.722789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.802667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.687470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.855893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.896735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.153101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.189568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.575842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.640550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.375370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.250294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.556810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.244630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.215879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.885116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.708758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.574857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.601500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.301952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.288691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.229046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.262771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.312187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.202800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.206450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.180947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.653041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.739826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.635697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.397137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.522353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.710058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.971180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.795596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.754508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.480618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.578661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.543532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.556077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.479826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.546637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.404454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.548796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.580293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.503093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.330917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.406197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.192981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.245563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.355730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.456160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.751210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.101212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.150328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.306804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.443735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.583729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.037104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.641576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.475160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.495972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.461757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.425751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.219899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.193403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.033507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.197549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.146447] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.209550] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5438.232590] Lustre: Skipped 128 previous similar messages [ 5438.262768] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5438.284325] Lustre: Skipped 128 previous similar messages [ 5438.287762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.284728] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5439.291993] Lustre: Skipped 128 previous similar messages [ 5439.297874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.306155] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5439.309053] Lustre: Skipped 128 previous similar messages [ 5440.525078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.631779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.512115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.588567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.706692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.514554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.734702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.495882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.211716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.827631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.800639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.657710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.524790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.397800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.421084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.226879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.439362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.729827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.625469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.855413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.936959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.993526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.415298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.341604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.715924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.944193] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.946067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.736250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.583643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.214980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.110183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.873732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.844798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.551683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.465198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.489186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.996651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.976793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.060326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.872186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.779856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.513349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.501866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.758565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.660360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.843899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.941598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.934550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.981974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.931502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.197363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.873716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.344877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.661199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.572891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.837117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.908429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.411883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.389084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.582513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.812669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.104747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.061412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.085481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.883233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.731050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.519604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.458908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.441671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.324400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.228475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.120646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.006774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.819499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.712896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.490824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.081688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.922456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.580445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.490904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.625544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.531640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.400069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.663725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.732058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.111311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.873410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.870768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.956848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.342467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.787418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.008769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.275663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.213499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.479969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.395571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.449883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.532068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.279131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.050984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.891898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.727356] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5542.737518] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5542.754875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.681534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.826547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.944787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.941637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.306765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.254989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.352833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.840278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.887456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.954774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.707992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.528060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.467740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.307486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.349140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.423178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.459892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.457551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.323455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.218088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.114281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.250485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.084561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.885961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.860258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.018542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.892893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.899065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.777832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.696336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.650519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.615435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.465145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.421794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.410048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.363220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.145704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.092548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.258779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.658630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.659083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.726523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.629639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.433191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.286587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.085699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.053762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.852985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.903136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.888589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.846239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.279648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.502977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.657085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.742593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.655553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.373613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.365811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.351519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.188165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.156786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.363318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.646422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.660594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.468367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.409086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.386926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.182804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.023149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.707730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.462196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.297995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.308365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.504492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.287674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.073015] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.961294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.218616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.090950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.792698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.827896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.732665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.979987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.839244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.771854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.708454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.808678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.717456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.689386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.447818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.398830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.615141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.710412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.737758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.950032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.894277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.958287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.947962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.998285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.102118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.198130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.335522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.389713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.687916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.769155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.721897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.582745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.461556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.544591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.805824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.016061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.886394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.620927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.633562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.530471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.682139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.869691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.833759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.898463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.761823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.850650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.331790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.288473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.581419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.384515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.889528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.057029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.280246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.062824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.524413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.595506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.494569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.671588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.482540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.777537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.800851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.407900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.133504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.971737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.989226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.040232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.929599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.854549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.061858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.994488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.833916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.549527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.492338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.526320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.402509] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5694.407506] Lustre: Skipped 252 previous similar messages [ 5694.429185] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5694.439906] Lustre: Skipped 252 previous similar messages [ 5694.444484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.353490] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5695.357874] Lustre: Skipped 252 previous similar messages [ 5695.361696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.378725] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5695.382943] Lustre: Skipped 252 previous similar messages [ 5696.254818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.146487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.978077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.959446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.860905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.690523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.614595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.475658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.411736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.253026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.090858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.907490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.750524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.508538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.377681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.220783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.121745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.053931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.077175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.016741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.105787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.056652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.060535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.761498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.862376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.900438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.821231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.605528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.452656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.240750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.161683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.080759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.201635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.001409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.890901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.760045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.865832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.961164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.805577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.746654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.462542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.583805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.565654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.539482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.250909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.157370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.163587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.139400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.210103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.094641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.954661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.967098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.925101] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.734669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.769796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.157575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.353047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.461355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.533640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.317458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.062600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.825699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.808274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.619843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.489106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.446637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.454574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.281360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.087185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.018942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.906961] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5760.913773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.915895] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5761.685533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.614378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.475471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.398348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.172392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.057317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.966728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.795758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.745899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.669662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.426531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.348363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.501252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.505952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.575786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.787038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.841570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.804944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.850338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.145240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.217473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.432928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.452601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.533553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.353580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.306790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.234152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.216146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.005189] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.286463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.476607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.222347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.295987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.241934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.755838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.043429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.247275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.130801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.156780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.100669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.189547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.113943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.047500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.747503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.616596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.428352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.506597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.630619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.610467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.465149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.378052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.458332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.343544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.197952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.908061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.812574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.612191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.522595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.189780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.046954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.716667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.538142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.293299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.163168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.908434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.779432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.718888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.588373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.283649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.344635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.127962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.039502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.808803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.845592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.703964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.654325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.616922] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.591095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.568288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.708742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.704827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.794927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.712205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.718649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.736338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.552363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.712308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.902350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.785137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.788251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.433422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.337521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.375943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.440421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.288752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.359740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.361878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.555574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.691250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.475315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.745041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.709333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.850626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.987864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.917373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.044814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.874900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.911248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.134527] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5866.149163] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5866.163451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.300828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.509214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.486223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.267351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.146363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.425591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.559098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.799701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.805499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.746985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.681329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.956243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.900995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.942792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.883971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.857067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.834889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.645220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.528457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.450519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.488403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.486126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.272274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.341763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.256231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.347834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.261672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.967549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.815539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.045763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.252621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.765183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.089764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.418431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.449882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.317232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.150925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.193221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.213059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.320855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.176064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.173229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.959377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.750681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.359336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.164591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.967522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.960505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.863292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.719804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.483154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.514884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.610588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.391685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.314105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.201376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.181356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.885980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.920886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.718982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.590865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.502984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.364691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.339799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.128988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.915243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.890914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.895154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.954112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.947734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.804831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.714597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.788515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.441306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.355712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.394963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.296607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.306321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.397551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.197952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.267303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.182906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.552227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.680270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.688556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.491978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.379745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.276671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.977459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.940659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.746096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.659293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.528681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.349167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.232897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.036350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.722724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.498258] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.358672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.092790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.861393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.855711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.707692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.571874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.403406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.229570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.116291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.038512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.995208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.125750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.059607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.044452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.988900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.812928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.017295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.818425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.151054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.234865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.977853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.852546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.933551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.147210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.483709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.663044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.996711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.274143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.604094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.338527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.169801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.851904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.682326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.682637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.678801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.864548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.973733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.990393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.924464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.962752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.891675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.841067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.709142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.618786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.703915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.447075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.348353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.379068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.209779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.039292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.230716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.957555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.020081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.126393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.133696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.170861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.983486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.998643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.720795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.638765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.698855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.644230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.459810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.323856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.186567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.282101] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6023.297064] LustreError: 172128:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6023.299985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.180123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.100695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.030073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.183882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.974592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.944916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.745556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.647528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.711927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.714321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.519279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.365621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.213844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.051654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.735971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.563720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.375778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.310795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.128508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.108281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.861278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.852271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.800980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.787238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.488831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.327934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.083248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.857226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.727904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.687212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.681584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.534836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.372051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.306526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.029868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.827314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.531932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.290224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.084937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.839411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.655812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.601223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.413048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.503583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.195908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.004278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.700687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.452320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.535604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.498247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.575613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.509483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.684048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.544985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.295413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.073603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.029789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.044178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.090337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.932152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.742323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.657319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.555768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.327341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.327268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.986780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.072257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.939888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.923887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.843198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.694565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.334948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.264625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.160327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.186864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.089897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.296384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6092.569548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.517559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.478538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.324977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.110349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.942844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.794360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.514409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.391795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.097166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.980654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.918814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.764833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.541925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.393880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.212452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.081690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6107.049670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.101644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.018523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.070390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.885923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.683671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.616388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.564311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.350287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.448663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.260438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.992964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.775787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.923851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.919679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.145457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.039150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.248225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.160829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.358655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.355451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.471913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.387416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.691836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.843040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.958086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.834313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.443151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.827245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.750473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.980658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.922791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.860932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.692380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.557916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.661713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.363791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.123334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.727644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.528895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.202749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.048725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.627849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.633844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.575977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.279702] LustreError: 173929:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6483.302675] LustreError: 172128:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6939.105178] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6939.116536] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6939.132204] Lustre: Skipped 484 previous similar messages [ 6939.147750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6942.564523] Lustre: server umount lustre-OST0000 complete [ 6946.794259] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6946.807310] LustreError: Skipped 4 previous similar messages [ 6946.815780] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6946.826191] Lustre: Skipped 1 previous similar message [ 6949.860098] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6949.867287] Lustre: Skipped 1 previous similar message [ 6953.354859] Lustre: server umount lustre-MDT0000 complete [ 6954.978254] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6955.004680] LustreError: 172108:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6956.677841] LustreError: 172089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786750834 with bad export cookie 9187204278892503966 [ 6956.683385] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6956.689170] LustreError: 172089:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6956.930757] Lustre: server umount lustre-MDT0001 complete [ 6967.296740] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 6970.601792] Key type lgssc unregistered [ 6970.926803] LNet: 227336:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6970.929963] LNetError: 227336:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6970.941522] LNet: Removed LNI 192.168.201.124@tcp [ 6971.695451] Key type .llcrypt unregistered [ 6971.696988] Key type ._llcrypt unregistered [ 6998.233832] Key type ._llcrypt registered [ 6998.236381] Key type .llcrypt registered [ 6998.451156] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7000.206265] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7000.251498] alg: No test for adler32 (adler32-zlib) [ 7001.291640] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 7001.518506] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 7003.217902] Key type lgssc registered [ 7004.471826] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7012.414473] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7012.431903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7013.880760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7018.208493] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7024.101731] LustreError: 228753:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7024.129584] LustreError: 228753:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7026.984866] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7027.499027] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7032.233491] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7040.612329] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7047.050401] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7054.195592] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7054.468033] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7058.574813] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7061.355392] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7068.861802] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7076.750887] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7079.104699] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7079.119463] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7079.136045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7079.156673] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7079.158432] LustreError: 228773:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7082.469817] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7082.488740] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7082.504335] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7084.518738] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7084.522069] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7085.946073] Lustre: lustre-OST0000: Not available for connect from 192.168.201.24@tcp (stopping) [ 7085.952979] Lustre: Skipped 1 previous similar message [ 7087.245973] Lustre: server umount lustre-OST0000 complete [ 7099.240121] LustreError: 228773:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7123.273377] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7123.885349] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7124.907928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7124.916812] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7132.134771] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7135.162971] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7135.209877] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7140.513471] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7141.898345] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 7146.949192] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff96f042984000.ost_server_uuid 50 [ 7148.326851] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff96f042984000.ost_server_uuid in IDLE state after 0 sec [ 7153.633507] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7153.644062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7153.662827] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7155.685291] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7159.709619] Lustre: server umount lustre-OST0000 complete [ 7165.921201] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7165.925164] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7165.956805] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7165.956814] Lustre: Skipped 2 previous similar messages [ 7165.986679] Lustre: Skipped 2 previous similar messages [ 7171.042867] LustreError: 229482:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7171.059570] LustreError: 229482:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7171.553874] Lustre: server umount lustre-MDT0000 complete [ 7175.263905] LustreError: 228736:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751052 with bad export cookie 17441151332508027802 [ 7175.279598] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7176.006459] Lustre: server umount lustre-MDT0001 complete [ 7187.792394] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 7190.935612] Key type lgssc unregistered [ 7191.223688] LNet: 233963:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7191.248154] LNetError: 233963:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7191.272326] LNet: Removed LNI 192.168.201.124@tcp [ 7192.343492] Key type .llcrypt unregistered [ 7192.346808] Key type ._llcrypt unregistered [ 7226.964566] Key type ._llcrypt registered [ 7226.966130] Key type .llcrypt registered [ 7227.129328] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7228.409020] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7228.586812] alg: No test for adler32 (adler32-zlib) [ 7229.775287] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 7230.208811] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 7231.960702] Key type lgssc registered [ 7233.995726] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7244.671987] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7244.705765] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7246.183281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7251.136477] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7255.540689] LustreError: 235695:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7255.558698] LustreError: 235695:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7260.649756] LustreError: 235694:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7261.122064] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7261.643731] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7267.626707] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7270.832851] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7277.752517] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7278.061085] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7285.045241] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7291.389458] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7294.575337] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7294.826389] Lustre: lustre-OST0001: new disk, initializing [ 7294.830726] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7294.835686] Lustre: 238001:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7294.958730] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7301.424278] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7303.805307] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7303.813911] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7303.908761] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7312.500666] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7320.849321] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7334.882854] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7334.904293] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7334.908889] Lustre: Skipped 2 previous similar messages [ 7334.919172] Lustre: Skipped 3 previous similar messages [ 7340.003071] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7340.010528] Lustre: Skipped 2 previous similar messages [ 7340.492826] Lustre: server umount lustre-MDT0000 complete [ 7343.014016] LustreError: 236422:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7344.341867] LustreError: 235675:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751221 with bad export cookie 6258109158649521821 [ 7344.365563] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7344.848245] Lustre: server umount lustre-MDT0001 complete [ 7359.188672] Lustre: server umount lustre-OST0000 complete [ 7373.531469] Lustre: server umount lustre-OST0001 complete [ 7397.248320] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7408.199110] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7408.669989] LustreError: 241119:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7408.676197] LustreError: 241119:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7408.756277] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7412.839477] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7414.250078] LustreError: 241120:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7422.929989] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7423.469477] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7428.648927] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7431.883398] Lustre: 242261:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7439.726697] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7440.447862] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7447.623866] LustreError: 242615:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7447.667171] LustreError: 242615:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7448.464700] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7453.690791] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 7457.311582] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7462.899346] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7464.327866] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7473.852615] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7477.709577] Lustre: 244130:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7490.021167] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7490.029434] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7490.043084] Lustre: Skipped 1 previous similar message [ 7490.047129] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7491.555085] Lustre: mylustre-MDT0000-lwp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7491.602060] Lustre: Skipped 2 previous similar messages [ 7496.679168] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7496.689127] Lustre: Skipped 7 previous similar messages [ 7501.796393] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7501.814517] Lustre: Skipped 3 previous similar messages [ 7502.308165] Lustre: mylustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7502.636822] Lustre: server umount mylustre-MDT0000 complete [ 7506.639581] LustreError: 244331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751384 with bad export cookie 6258109158649524621 [ 7506.644299] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7506.652120] LustreError: 244331:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7506.915710] LustreError: 242292:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7506.918405] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7506.926643] LustreError: 242292:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 7506.948807] Lustre: Skipped 1 previous similar message [ 7507.170737] Lustre: server umount mylustre-MDT0001 complete [ 7513.844425] Lustre: server umount mylustre-OST0000 complete [ 7518.031602] Lustre: server umount mylustre-OST0001 complete [ 7541.985360] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7554.421677] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7554.796482] LustreError: 246439:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7554.818277] LustreError: 246439:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7554.891925] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7554.903562] Lustre: Skipped 1 previous similar message [ 7560.084982] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7570.000858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7575.262748] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7578.356971] Lustre: 247582:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7584.945901] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7591.812098] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7599.608710] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 7601.162049] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7606.795561] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7609.069835] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7618.040784] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7623.209330] Lustre: 249452:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7632.373290] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7632.376616] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7632.399761] Lustre: Skipped 2 previous similar messages [ 7632.409115] Lustre: Skipped 3 previous similar messages [ 7634.767562] Lustre: server umount tfs-MDT0000 complete [ 7637.482390] LustreError: 246435:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7637.501872] LustreError: 246435:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 7639.152858] LustreError: 246415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751516 with bad export cookie 6258109158649526511 [ 7639.156317] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7639.167303] LustreError: 246415:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7639.569156] Lustre: server umount tfs-MDT0001 complete [ 7655.593277] Lustre: server umount tfs-OST0000 complete [ 7658.978421] Lustre: 234956:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786751520/real 1786751520] req@ffff92e142108700 x1873544328983424/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786751536 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 7659.020620] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7659.037644] Lustre: Skipped 1 previous similar message [ 7660.719268] Lustre: server umount tfs-OST0001 complete [ 7686.575247] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7699.537549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7700.308744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7700.313761] Lustre: Skipped 3 previous similar messages [ 7705.554366] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7716.326584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7722.998876] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7727.237707] Lustre: 252844:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7736.099249] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7744.316944] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7744.772838] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 7752.419954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7757.830340] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7761.101477] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7770.851762] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7775.132442] Lustre: 254715:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7783.751757] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:54:18 (1786751658) [ 7803.873462] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7803.888249] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7803.900794] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7803.922656] Lustre: Skipped 3 previous similar messages [ 7803.940798] Lustre: Skipped 3 previous similar messages [ 7807.470674] Lustre: server umount lustre-MDT0000 complete [ 7809.004368] LustreError: 251703:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7809.050331] LustreError: 251703:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 7811.849859] LustreError: 252448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751689 with bad export cookie 6258109158649528170 [ 7811.865682] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7811.882676] LustreError: 252448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7818.362678] Lustre: server umount lustre-MDT0001 complete [ 7829.712505] Lustre: server umount lustre-OST0000 complete [ 7840.384884] Lustre: server umount lustre-OST0001 complete [ 7849.434816] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 7861.559478] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 7898.589731] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 7898.589731] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7898.589731] [ 7898.636072] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7898.983341] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7899.024913] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7899.159265] Lustre: lustre-MDT0000: new disk, initializing [ 7899.263788] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7899.267781] Lustre: Skipped 3 previous similar messages [ 7899.292993] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7904.276711] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7914.074082] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 7914.074082] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 7914.074082] [ 7914.100159] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 7914.270740] Lustre: 257902:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7914.374109] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7914.382238] Lustre: Skipped 1 previous similar message [ 7914.490327] Lustre: lustre-MDT0001: new disk, initializing [ 7914.726223] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7914.735630] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7919.495816] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7930.210038] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7930.549522] Lustre: lustre-OST0000: new disk, initializing [ 7930.556714] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7930.566691] Lustre: 259484:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7932.579260] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7932.589851] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7932.784710] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7939.067208] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7950.519857] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7950.774630] Lustre: lustre-OST0001: new disk, initializing [ 7950.786214] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7950.807362] Lustre: 260354:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7952.444788] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7952.455799] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7952.592187] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7957.447188] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 7968.637694] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:57:23 (1786751843) [ 7982.528721] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:57:37 (1786751857) [ 7998.953356] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7998.958678] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7998.969526] Lustre: Skipped 3 previous similar messages [ 7998.978805] Lustre: Skipped 5 previous similar messages [ 8004.710390] Lustre: server umount lustre-OST0000 complete [ 8015.370455] Lustre: server umount lustre-MDT0000 complete [ 8019.112879] LustreError: 257892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786751896 with bad export cookie 6258109158649529731 [ 8019.125626] LustreError: 257892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8019.131645] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8025.495730] Lustre: server umount lustre-MDT0001 complete [ 8037.768167] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 8108.437676] Lustre: server umount lustre-OST0001 complete [ 8115.900982] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 8131.583836] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8168.830382] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8169.031677] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8169.076906] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8169.131711] Lustre: lustre-MDT0000: new disk, initializing [ 8169.174539] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8169.177741] Lustre: Skipped 3 previous similar messages [ 8169.187779] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8174.780189] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8188.643578] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8188.760639] Lustre: 265151:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8188.803900] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8188.807559] Lustre: Skipped 1 previous similar message [ 8188.873357] Lustre: lustre-MDT0001: new disk, initializing [ 8188.995949] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8189.005951] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8193.590244] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8203.577986] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8210.287510] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8217.502443] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8217.740740] Lustre: 267131:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8219.456855] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8219.578776] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8224.990929] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8234.235422] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8245.218149] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8245.243895] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8245.255764] Lustre: Skipped 5 previous similar messages [ 8245.265711] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8245.269603] Lustre: Skipped 10 previous similar messages [ 8247.963507] Lustre: server umount lustre-OST0000 complete [ 8255.482838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8255.492732] LustreError: Skipped 1 previous similar message [ 8258.758745] Lustre: server umount lustre-MDT0000 complete [ 8260.589423] LustreError: 265161:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8260.628313] LustreError: 265161:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 8262.989885] LustreError: 265141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786752140 with bad export cookie 6258109158649531201 [ 8263.007647] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8263.014564] LustreError: 265141:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8263.519819] Lustre: server umount lustre-MDT0001 complete [ 8276.585965] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 8280.218074] Key type lgssc unregistered [ 8280.460645] LNet: 268897:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8280.475783] LNetError: 268897:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8280.500559] LNet: Removed LNI 192.168.201.124@tcp [ 8281.477194] Key type .llcrypt unregistered [ 8281.478706] Key type ._llcrypt unregistered [ 8310.041688] Key type ._llcrypt registered [ 8310.044481] Key type .llcrypt registered [ 8310.141929] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8311.761136] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8311.801469] alg: No test for adler32 (adler32-zlib) [ 8312.890522] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 8313.149122] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 8314.872972] Key type lgssc registered [ 8316.055571] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8323.975595] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8324.011661] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8325.510913] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8329.759419] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8335.841832] LustreError: 270359:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8335.867554] LustreError: 270359:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8337.983169] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8338.314152] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8342.385841] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8349.793932] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8357.160937] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8364.971704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8365.222918] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8371.387846] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8372.497136] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8380.265548] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8381.117116] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8382.660335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_create to node 0@lo failed: rc = -107 [ 8382.663742] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8382.669142] LustreError: 271090:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0000-osc-MDT0001: cannot cleanup orphans: rc = -107 [ 8382.674122] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8387.041581] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8387.047928] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8387.066462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8388.860115] Lustre: server umount lustre-OST0000 complete [ 8393.186148] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8393.189505] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8393.198108] Lustre: Skipped 1 previous similar message [ 8393.204864] Lustre: Skipped 3 previous similar messages [ 8398.308117] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8398.314039] Lustre: Skipped 1 previous similar message [ 8399.169710] Lustre: server umount lustre-MDT0000 complete [ 8402.585937] LustreError: 270340:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786752280 with bad export cookie 5155547769231181201 [ 8402.596650] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8402.909963] Lustre: server umount lustre-MDT0001 complete [ 8413.001453] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 8415.553462] Key type lgssc unregistered [ 8416.004921] LNet: 273944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8416.022867] LNetError: 273944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8416.053753] LNet: Removed LNI 192.168.201.124@tcp [ 8417.171290] Key type .llcrypt unregistered [ 8417.176085] Key type ._llcrypt unregistered [ 8435.273415] Key type ._llcrypt registered [ 8435.276445] Key type .llcrypt registered [ 8435.403976] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8436.558070] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8436.810229] alg: No test for adler32 (adler32-zlib) [ 8437.884486] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 8438.153817] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 8439.872392] Key type lgssc registered [ 8440.805250] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8449.867252] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8449.915141] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8465.440445] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8470.560771] LustreError: 275139:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.124@tcp: failed processing log, type 4: rc = -110 [ 8497.248527] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8502.875811] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8510.386448] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8512.560467] Lustre: server umount lustre-OST0000 complete [ 8529.816765] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 8532.936728] Key type lgssc unregistered [ 8533.181416] LNet: 276775:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8533.194341] LNetError: 276775:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8533.211183] LNet: Removed LNI 192.168.201.124@tcp [ 8533.979430] Key type .llcrypt unregistered [ 8533.981707] Key type ._llcrypt unregistered [ 8572.000410] Key type ._llcrypt registered [ 8572.002543] Key type .llcrypt registered [ 8572.096234] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8573.111810] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8573.329100] alg: No test for adler32 (adler32-zlib) [ 8574.423392] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 8574.709723] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 8576.424193] Key type lgssc registered [ 8578.072905] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8592.666503] loop0: detected capacity change from 0 to 204800000 [ 8592.939731] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8592.948301] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8593.402136] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8594.242538] loop0: detected capacity change from 0 to 204800000 [ 8594.432104] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8594.439577] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8594.634523] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8595.595991] loop0: detected capacity change from 0 to 204800000 [ 8595.775490] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8595.787838] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8595.957656] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8597.164825] loop0: detected capacity change from 0 to 204800000 [ 8597.388624] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8598.671788] loop0: detected capacity change from 0 to 204800000 [ 8598.702944] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8603.552271] loop0: detected capacity change from 0 to 204800000 [ 8603.614943] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8608.308194] loop0: detected capacity change from 0 to 204800000 [ 8608.347722] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8613.284472] loop0: detected capacity change from 0 to 204800000 [ 8613.332261] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8617.425424] loop0: detected capacity change from 0 to 204800000 [ 8617.556124] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8617.625792] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8617.675580] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8617.693597] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8617.712664] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8617.739411] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8617.757647] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8617.772559] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8617.781806] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8617.806848] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8617.816757] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8617.829316] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8617.836722] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8617.846237] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8617.853530] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8617.860841] Lustre: 280194:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8623.883216] Lustre: server umount MGS complete [ 8625.201532] loop0: detected capacity change from 0 to 204800000 [ 8625.262569] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8625.806281] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8625.848378] Lustre: lustre-MDT0000: Aborting client recovery [ 8626.738100] loop1: detected capacity change from 0 to 204800000 [ 8626.900688] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8627.033929] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8627.053217] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8627.073586] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8627.094637] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8627.106731] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8627.114874] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8627.123794] Lustre: 280625:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8627.145241] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8627.322154] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8627.369232] Lustre: lustre-MDT0001: Aborting client recovery [ 8628.623231] loop2: detected capacity change from 0 to 204800000 [ 8628.783578] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8628.893405] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8628.918183] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8628.950854] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8628.976752] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8628.995440] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8629.031074] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8629.058129] Lustre: 280703:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8629.251841] Lustre: Found index 0 for lustre-OST0000, updating log [ 8629.322418] Lustre: 280720:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8629.496472] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8629.539742] Lustre: lustre-OST0000: Aborting recovery [ 8630.567541] loop3: detected capacity change from 0 to 204800000 [ 8630.776173] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8630.859940] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8630.874716] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8630.894550] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8630.914607] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8630.935621] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8630.951210] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8630.964991] Lustre: 280786:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8631.061795] Lustre: Found index 1 for lustre-OST0001, updating log [ 8631.122827] Lustre: 280791:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8631.205539] Lustre: lustre-OST0001: Aborting recovery [ 8636.331614] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8636.336506] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8636.397534] LustreError: 280569:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 8636.406389] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8636.406639] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8637.348171] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8637.350555] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8639.321651] LustreError: 280536:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1094us: rc = -2 [ 8639.832694] LustreError: 280536:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 395us: rc = -2 [ 8639.855828] LustreError: 280536:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 22 previous similar messages [ 8652.768964] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8652.771519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8652.795962] Lustre: Skipped 2 previous similar messages [ 8652.802654] Lustre: Skipped 1 previous similar message [ 8657.889618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8657.915327] Lustre: Skipped 5 previous similar messages [ 8658.892665] Lustre: server umount lustre-MDT0000 complete [ 8660.030384] LustreError: 280518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786752537 with bad export cookie 14487609207245239565 [ 8660.042310] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8660.045135] LustreError: 280518:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8663.009165] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8663.015237] LustreError: 280534:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8663.020903] Lustre: Skipped 1 previous similar message [ 8663.022621] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8663.072936] LustreError: 280534:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8665.057415] LustreError: 280534:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8665.058169] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8665.094667] Lustre: Skipped 2 previous similar messages [ 8666.684584] Lustre: server umount lustre-MDT0001 complete [ 8673.895556] Lustre: server umount lustre-OST0000 complete [ 8689.120735] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8689.334082] Lustre: server umount lustre-OST0001 complete [ 8697.100054] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 20:09:31 (1786752571) [ 8724.092776] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 8734.190422] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 8777.496193] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8777.743495] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8777.775905] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8777.831765] Lustre: lustre-MDT0000: new disk, initializing [ 8777.892785] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8777.897421] Lustre: Skipped 1 previous similar message [ 8777.911401] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8782.036250] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8792.479805] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8792.615768] Lustre: 284864:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8792.695925] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8792.710142] Lustre: Skipped 1 previous similar message [ 8792.782989] Lustre: lustre-MDT0001: new disk, initializing [ 8792.861943] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8792.894503] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8792.912178] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8797.213342] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8806.059422] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8812.047254] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8818.397837] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8818.699164] Lustre: lustre-OST0000: new disk, initializing [ 8818.703511] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8818.714935] Lustre: 286832:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8818.824175] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8824.801031] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8826.906732] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8826.939773] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8827.049058] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8833.375328] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8838.371783] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8850.882660] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8861.648226] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8868.214591] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8914.402632] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8914.408439] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8914.424145] LustreError: Skipped 1 previous similar message [ 8914.426876] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8914.498190] Lustre: Skipped 2 previous similar messages [ 8924.646148] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8924.663037] Lustre: Skipped 4 previous similar messages [ 8928.224135] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8928.699562] Lustre: server umount lustre-OST0000 complete [ 8934.892215] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8934.904340] Lustre: Skipped 1 previous similar message [ 8939.309883] Lustre: server umount lustre-MDT0000 complete [ 8940.001154] LustreError: 284876:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8940.021204] LustreError: 284876:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8943.019993] LustreError: 284875:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8943.037721] LustreError: 284875:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8943.191524] LustreError: 284855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786752820 with bad export cookie 14487609207245242267 [ 8943.207639] LustreError: 284855:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 8943.207970] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8943.543628] Lustre: server umount lustre-MDT0001 complete [ 8952.097435] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8956.980676] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8961.808417] Lustre: server umount MGS complete [ 8973.308606] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8973.704544] LustreError: 290894:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8973.722156] LustreError: 290907:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8973.767551] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8978.286093] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8984.038277] LustreError: 290895:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8984.058660] LustreError: 290895:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8986.487854] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8991.213215] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 8998.522353] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9004.893609] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9012.599419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9012.883750] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9012.890319] Lustre: Skipped 1 previous similar message [ 9018.794474] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9025.859819] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9058.283362] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9058.325517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9058.328389] Lustre: Skipped 2 previous similar messages [ 9063.725492] Lustre: server umount lustre-OST0000 complete [ 9069.026780] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9069.030076] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9069.059019] Lustre: Skipped 1 previous similar message [ 9083.361596] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9083.504211] Lustre: server umount lustre-MDT0000 complete [ 9083.873599] LustreError: 293730:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9083.890163] LustreError: 293730:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9087.003943] LustreError: 290876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786752964 with bad export cookie 14487609207245244122 [ 9087.008536] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9087.016855] LustreError: 290876:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9087.385313] Lustre: server umount lustre-MDT0001 complete [ 9098.501465] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 9102.157958] Key type lgssc unregistered [ 9102.573820] LNet: 294842:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9102.587770] LNetError: 294842:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9102.623750] LNet: Removed LNI 192.168.201.124@tcp [ 9103.589069] Key type .llcrypt unregistered [ 9103.590610] Key type ._llcrypt unregistered [ 9143.169750] Key type ._llcrypt registered [ 9143.172133] Key type .llcrypt registered [ 9143.302858] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 9159.543264] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9160.943584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9161.127683] alg: No test for adler32 (adler32-zlib) [ 9162.294541] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 9162.538905] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 9164.200168] Key type lgssc registered [ 9165.285389] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9197.733796] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9197.760144] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9199.041717] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9199.070426] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9199.131492] Lustre: lustre-MDT0000: new disk, initializing [ 9199.206790] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9199.222585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9202.906498] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9213.912812] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9214.013250] Lustre: 298806:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9214.050792] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9214.054125] Lustre: Skipped 1 previous similar message [ 9214.126730] Lustre: lustre-MDT0001: new disk, initializing [ 9214.209888] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9214.254781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9214.265903] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9219.965804] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9229.064467] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9235.563281] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9242.522324] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9242.857938] Lustre: lustre-OST0000: new disk, initializing [ 9242.862122] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9242.871551] Lustre: 300775:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9242.973371] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9250.089945] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9251.375625] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9251.378806] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9251.460717] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9259.447524] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9268.129769] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9278.896936] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9284.760346] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9333.234532] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9333.236665] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9333.260096] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9333.279783] Lustre: Skipped 1 previous similar message [ 9337.608518] Lustre: server umount lustre-OST0000 complete [ 9342.436648] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9342.443756] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9342.452222] Lustre: Skipped 1 previous similar message [ 9342.458126] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9343.458306] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9343.461083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9343.477572] Lustre: Skipped 1 previous similar message [ 9347.057574] Lustre: server umount lustre-MDT0000 complete [ 9350.567649] LustreError: 298797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786753228 with bad export cookie 11380878144386417247 [ 9350.569683] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9350.573339] LustreError: 298797:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9350.873466] Lustre: server umount lustre-MDT0001 complete [ 9358.605943] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9362.596132] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9367.518593] Lustre: server umount MGS complete [ 9377.737035] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9378.202750] LustreError: 304638:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9378.210751] LustreError: 304638:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9378.217834] LustreError: 304651:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9378.319096] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9382.386253] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9383.394315] LustreError: 304639:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9387.502396] LustreError: 304638:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9391.336557] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9391.558285] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9395.719238] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9402.897843] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9408.626246] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9415.141485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9415.453248] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9421.082989] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9427.282970] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9452.514738] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9452.519970] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9452.527324] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9454.392996] Lustre: server umount lustre-OST0000 complete [ 9458.151388] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9458.157693] LustreError: Skipped 1 previous similar message [ 9458.159945] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9458.165739] Lustre: Skipped 1 previous similar message [ 9458.168814] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9458.175558] Lustre: Skipped 2 previous similar messages [ 9464.797512] Lustre: server umount lustre-MDT0000 complete [ 9468.913639] LustreError: 304639:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9468.936871] LustreError: 304639:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9469.887421] LustreError: 304618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786753347 with bad export cookie 11380878144386419088 [ 9469.901512] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9469.912471] LustreError: 304618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9470.424344] Lustre: server umount lustre-MDT0001 complete [ 9480.942203] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [ 9484.226925] Key type lgssc unregistered [ 9484.586717] LNet: 308589:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9484.598510] LNetError: 308589:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9485.671408] LNet: Removed LNI 192.168.201.124@tcp [ 9486.462039] Key type .llcrypt unregistered [ 9486.467231] Key type ._llcrypt unregistered [ 9505.174670] Key type ._llcrypt registered [ 9505.177075] Key type .llcrypt registered [ 9505.273483] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [ 9516.812429] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9517.724525] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9517.869380] alg: No test for adler32 (adler32-zlib) [ 9518.939888] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [ 9519.153710] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [ 9520.824553] Key type lgssc registered [ 9521.822795] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9544.734874] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [ 9557.334946] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9557.372478] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9558.581771] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9558.612417] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9558.675264] Lustre: lustre-MDT0000: new disk, initializing [ 9558.757991] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9558.778319] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9563.072354] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [ 9572.245494] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9572.468284] Lustre: lustre-OST0000: new disk, initializing [ 9572.471725] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9572.475106] Lustre: Skipped 1 previous similar message [ 9572.478621] Lustre: 312617:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9572.529497] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9574.282634] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9574.297669] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9574.332184] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 9577.789594] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10352.611124] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10352.626526] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10355.614189] Lustre: server umount lustre-MDT0000 complete [10359.626407] LustreError: 311805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786754237 with bad export cookie 18316898627644205550 [10359.631789] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10359.637933] LustreError: 311805:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10359.942134] Lustre: server umount lustre-OST0000 complete [10387.152111] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [10398.218154] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [10437.809834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10438.265813] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10438.317906] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10438.436377] Lustre: lustre-MDT0000: new disk, initializing [10438.528638] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10438.551190] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10443.634446] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10453.408138] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10453.613179] Lustre: 316947:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10453.708802] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10453.712579] Lustre: Skipped 1 previous similar message [10453.835945] Lustre: lustre-MDT0001: new disk, initializing [10453.907894] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10453.942270] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10453.953692] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10457.497361] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10466.957346] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10472.503594] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10479.389527] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10479.962865] Lustre: lustre-OST0000: new disk, initializing [10479.965129] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10479.979468] Lustre: 318916:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10480.129840] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10486.867267] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10486.896849] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10487.066621] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10488.022698] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10498.910787] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10504.735789] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10505.040673] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10509.377260] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10509.632634] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10512.870279] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10512.882259] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10512.898097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10512.924975] LustreError: Skipped 1 previous similar message [10517.875580] Lustre: server umount lustre-OST0000 complete [10523.107351] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10523.130840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10523.150435] Lustre: Skipped 2 previous similar messages [10523.163994] Lustre: Skipped 2 previous similar messages [10527.685708] Lustre: server umount lustre-MDT0000 complete [10530.933347] LustreError: 316938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786754408 with bad export cookie 18316898627647146369 [10530.941780] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10530.945671] LustreError: 316938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10531.279430] Lustre: server umount lustre-MDT0001 complete [10534.479811] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10536.110677] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 20:40:11 (1786754411) [10541.970735] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10542.530450] LustreError: 321228:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10542.545787] LustreError: 321228:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10542.587480] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10547.597848] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10547.682831] LustreError: 321229:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10552.803919] LustreError: 321228:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10555.451108] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10555.929377] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10560.891171] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10567.602195] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10572.916529] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10579.840142] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10580.120469] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10586.971408] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10594.093954] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10601.306622] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10601.517462] Lustre: lustre-OST0001: new disk, initializing [10601.519816] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10601.523829] Lustre: 324064:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10601.614092] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10607.183362] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10610.737846] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10610.751310] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10610.807036] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10611.875764] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10611.888244] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10612.900758] LustreError: 324567:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10612.911792] LustreError: 324567:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10614.947467] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10614.967897] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10616.941663] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10617.156133] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10619.129296] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10619.142601] LustreError: 324557:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10623.479738] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff96f0663f5800.ost_server_uuid 50 [10625.192800] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff96f0663f5800.ost_server_uuid in FULL state after 0 sec [10627.367743] LustreError: 324567:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10627.375625] LustreError: 324567:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10632.164118] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff96f0663f5800.ost_server_uuid 50 [10634.117683] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff96f0663f5800.ost_server_uuid in FULL state after 0 sec [10641.936594] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10652.129329] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10652.136741] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10652.136979] LustreError: Skipped 1 previous similar message [10652.146563] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10652.169168] Lustre: Skipped 1 previous similar message [10652.211050] Lustre: Skipped 1 previous similar message [10656.815361] Lustre: server umount lustre-OST0001 complete [10657.249973] LustreError: 323049:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10657.275819] LustreError: 323049:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10662.371092] LustreError: 323051:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10662.419829] LustreError: 323051:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10665.443198] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10665.452258] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10665.466990] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10665.474201] Lustre: Skipped 1 previous similar message [10670.591179] Lustre: server umount lustre-OST0000 complete [10677.734715] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10677.754158] Lustre: Skipped 1 previous similar message [10677.757838] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10677.764866] Lustre: Skipped 3 previous similar messages [10680.898264] Lustre: server umount lustre-MDT0000 complete [10682.852585] LustreError: 321229:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10682.876228] LustreError: 321229:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10684.865262] LustreError: 321964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786754562 with bad export cookie 18316898627647147251 [10684.868340] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10684.873369] LustreError: 321964:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10685.282571] Lustre: server umount lustre-MDT0001 complete [10696.022871] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [10699.623933] Key type lgssc unregistered [10699.899973] LNet: 326758:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10699.905810] LNetError: 326758:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10700.970414] LNet: Removed LNI 192.168.201.124@tcp [10701.919868] Key type .llcrypt unregistered [10701.924544] Key type ._llcrypt unregistered [10729.151846] Key type ._llcrypt registered [10729.158710] Key type .llcrypt registered [10729.274937] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [10730.594573] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10730.630479] alg: No test for adler32 (adler32-zlib) [10731.763994] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [10732.034954] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [10733.704190] Key type lgssc registered [10735.027639] Lustre: Echo OBD driver; http://www.lustre.org/ [10743.595115] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10743.607367] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10745.050629] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10749.460339] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10754.537271] LustreError: 328185:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10754.557798] LustreError: 328185:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10757.965665] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10759.669706] LustreError: 328184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10764.787133] LustreError: 328184:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10765.152556] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10765.716487] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10770.433264] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10780.829834] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10781.069568] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10788.576468] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10790.316808] LustreError: 329863:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10790.330749] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10796.519067] LustreError: 329865:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10796.557581] LustreError: 329865:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10797.837840] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10805.379283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10805.783421] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10810.869933] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10812.151412] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [10825.025858] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10832.374537] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10832.610749] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10933.730678] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10933.732455] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10933.735908] LustreError: Skipped 1 previous similar message [10933.736749] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10933.765958] Lustre: Skipped 1 previous similar message [10938.854495] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10938.859584] Lustre: Skipped 2 previous similar messages [10939.852426] Lustre: server umount lustre-OST0001 complete [10943.977379] LustreError: 330313:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10944.000723] LustreError: 330313:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10949.089997] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10949.093834] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10949.101432] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10949.112460] Lustre: Skipped 1 previous similar message [10954.215363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10954.225846] Lustre: Skipped 2 previous similar messages [10959.335379] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10960.864139] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10960.981608] Lustre: server umount lustre-OST0000 complete [10965.481255] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10965.488731] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10969.569957] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10969.573176] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10969.605379] Lustre: Skipped 3 previous similar messages [10979.296216] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [10979.642421] Lustre: server umount lustre-MDT0000 complete [10979.821891] LustreError: 328181:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10979.840760] LustreError: 328181:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [10984.022837] LustreError: 328165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786754861 with bad export cookie 358639548957898427 [10984.033761] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10984.040509] LustreError: 328165:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10984.514791] Lustre: server umount lustre-MDT0001 complete [10995.996567] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [10998.742410] Key type lgssc unregistered [10999.087163] LNet: 333133:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10999.103254] LNetError: 333133:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11000.172540] LNet: Removed LNI 192.168.201.124@tcp [11001.110827] Key type .llcrypt unregistered [11001.113151] Key type ._llcrypt unregistered [11049.958867] Key type ._llcrypt registered [11049.960892] Key type .llcrypt registered [11050.050783] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11051.573283] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11051.599701] alg: No test for adler32 (adler32-zlib) [11052.682643] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [11052.862357] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [11054.536186] Key type lgssc registered [11055.650747] Lustre: Echo OBD driver; http://www.lustre.org/ [11065.353262] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11065.376898] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11066.854309] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11072.717353] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11077.091135] LustreError: 335432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11077.129988] LustreError: 335432:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11082.215765] LustreError: 335431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11083.013539] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11083.802514] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11090.130813] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11095.190284] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11105.476940] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11105.869874] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11108.985785] LustreError: 336922:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11111.910157] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11112.043914] LustreError: 336921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11112.070622] LustreError: 336921:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11112.081484] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11117.555436] LustreError: 336921:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11117.586122] LustreError: 336921:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11120.212821] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11120.659548] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11125.779665] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11127.355821] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11137.169424] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11145.828587] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11160.549049] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11160.560747] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11160.575918] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11161.576622] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11161.578809] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11161.586377] Lustre: Skipped 1 previous similar message [11161.613542] Lustre: Skipped 2 previous similar messages [11164.943487] Lustre: server umount lustre-MDT0000 complete [11166.699769] LustreError: 335431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11166.727476] LustreError: 335431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11168.798485] LustreError: 335414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786755046 with bad export cookie 14837404799823857764 [11168.813455] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11168.817753] LustreError: 335414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11169.307127] Lustre: server umount lustre-MDT0001 complete [11183.536152] Lustre: server umount lustre-OST0000 complete [11198.068481] Lustre: server umount lustre-OST0001 complete [11217.541653] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11228.113549] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11228.488997] LustreError: 340697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11228.542759] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11232.433726] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11240.259688] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11240.875646] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11244.433622] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11247.349236] Lustre: 341836:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11253.805633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11255.218993] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11261.061458] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11265.522482] LustreError: 342191:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11265.544164] LustreError: 342191:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11270.384534] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11270.794852] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11270.798796] Lustre: Skipped 1 previous similar message [11276.286619] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11276.737870] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11284.369778] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11287.349668] Lustre: 343707:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11291.618959] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11291.624274] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11291.641278] Lustre: Skipped 2 previous similar messages [11291.656274] Lustre: Skipped 5 previous similar messages [11296.737327] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11296.749441] Lustre: Skipped 3 previous similar messages [11297.434433] Lustre: server umount lustre-MDT0000 complete [11300.802253] LustreError: 340678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786755178 with bad export cookie 14837404799823859465 [11300.807461] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11300.817295] LustreError: 340678:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11301.212836] Lustre: server umount lustre-MDT0001 complete [11314.870215] Lustre: server umount lustre-OST0000 complete [11318.242507] Lustre: 334696:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786755179/real 1786755179] req@ffff92e148dddf80 x1873548337594752/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786755195 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11318.278890] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11318.289863] Lustre: Skipped 1 previous similar message [11318.981616] Lustre: server umount lustre-OST0001 complete [11323.396808] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11325.711784] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11328.177590] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 20:53:22 (1786755202) [11352.420287] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11361.941595] loop0: detected capacity change from 0 to 1099511627776 [11362.685451] SGI XFS with ACLs, security attributes, quota, no debug enabled [11362.690976] *** VALIDATE xfs *** [11362.702099] XFS (loop0): Mounting V5 Filesystem [11362.741525] XFS (loop0): Ending clean mount [11363.937633] loop1: detected capacity change from 0 to 18691697672192 [11483.620708] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [11492.936782] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11530.597374] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11530.930344] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11530.961377] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11531.051576] Lustre: lustre-MDT0000: new disk, initializing [11531.150661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11531.164774] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11535.751863] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11547.713667] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11547.852473] Lustre: 348843:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11547.908506] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11547.913425] Lustre: Skipped 1 previous similar message [11548.004797] Lustre: lustre-MDT0001: new disk, initializing [11548.190063] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11548.206113] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11552.186846] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11562.637658] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11569.597673] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11577.435136] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11577.709328] Lustre: lustre-OST0000: new disk, initializing [11577.712826] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11577.719356] Lustre: 350812:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11579.215864] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11579.231577] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11579.441996] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11584.263942] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11593.908309] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11598.800385] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11599.095872] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11606.809721] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11607.316893] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11615.205951] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11615.207265] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11615.208830] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11615.223210] LustreError: Skipped 1 previous similar message [11615.243976] Lustre: Skipped 1 previous similar message [11617.173273] Lustre: server umount lustre-OST0000 complete [11624.931277] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11624.950593] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11624.992303] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11624.994936] Lustre: Skipped 1 previous similar message [11627.699479] Lustre: server umount lustre-MDT0000 complete [11630.568513] LustreError: 348856:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11630.588893] LustreError: 348856:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11631.340872] LustreError: 352353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786755508 with bad export cookie 14837404799823861019 [11631.355575] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11631.357198] LustreError: 352353:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11631.706559] Lustre: server umount lustre-MDT0001 complete [11641.538499] XFS (loop0): Unmounting Filesystem [11647.022909] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 20:58:41 (1786755521) [11654.395418] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11655.082637] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11655.090230] Lustre: Skipped 2 previous similar messages [11660.583995] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11669.968026] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11675.849495] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11684.292082] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11690.077412] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11697.742107] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11704.774530] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11712.539887] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11724.772188] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11724.792123] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [11724.794294] Lustre: Skipped 1 previous similar message [11724.803367] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11724.823572] Lustre: Skipped 3 previous similar messages [11730.404121] Lustre: server umount lustre-OST0000 complete [11740.603228] Lustre: server umount lustre-MDT0000 complete [11743.945715] LustreError: 353354:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786755621 with bad export cookie 14837404799823861901 [11743.974594] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11744.273481] Lustre: server umount lustre-MDT0001 complete [11755.734890] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [11758.286785] Key type lgssc unregistered [11758.682670] LNet: 357004:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11758.696470] LNetError: 357004:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11758.720666] LNet: Removed LNI 192.168.201.124@tcp [11759.765339] Key type .llcrypt unregistered [11759.769260] Key type ._llcrypt unregistered [11788.536849] Key type ._llcrypt registered [11788.540345] Key type .llcrypt registered [11788.642503] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [11789.971828] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11790.048229] alg: No test for adler32 (adler32-zlib) [11791.223883] Lustre: Lustre: Build Version: 2.17.57_1_g59aa375 [11791.397147] LNet: Added LNI 192.168.201.124@tcp [8/256/0/180] [11793.068813] Key type lgssc registered [11793.969120] Lustre: Echo OBD driver; http://www.lustre.org/ [11801.420198] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11801.438767] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11802.898089] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11807.491655] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11813.349142] LustreError: 358583:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11813.362182] LustreError: 358583:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11816.863660] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11817.316969] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11821.750297] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11829.268130] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11835.241711] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11841.841972] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11842.019469] LustreError: 360401:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11842.310971] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11850.692741] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11859.272577] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11867.520354] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [11868.642783] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11868.650280] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11868.667583] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11869.168539] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11872.596925] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [11872.606162] Lustre: Skipped 2 previous similar messages [11873.830107] Lustre: server umount lustre-MDT0001 complete [11874.282354] LustreError: 360651:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11874.299138] LustreError: 360651:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [11877.708446] LustreError: 358578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11882.841647] LustreError: 360651:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11882.867869] LustreError: 360651:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11884.621684] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11884.845926] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11884.858387] Lustre: Found index 1 for lustre-MDT0001, updating log [11884.947646] Lustre: 358573:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11885.260430] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11890.524341] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [11890.686415] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11895.286565] LustreError: 362178:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11895.292342] LustreError: 362178:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11895.301169] LustreError: 358576:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11895.355479] LustreError: 362181:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -17 [11895.361263] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11895.361263] [12200.288318] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.24@tcp (stopping) [12202.986423] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12202.992244] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12205.287188] Lustre: server umount lustre-MDT0001 complete [12205.418433] LustreError: 358579:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.24@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12205.446188] LustreError: 358579:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12218.439489] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12218.599658] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12218.608789] Lustre: Found index 1 for lustre-MDT0001, updating log [12218.684773] Lustre: 358572:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12218.931029] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12223.669794] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12223.986736] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12225.567460] LustreError: 370728:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12225.573221] LustreError: 370728:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12225.589546] LustreError: 358576:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12225.595381] LustreError: 358576:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12225.691870] LustreError: 370737:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12225.696723] LustreError: 370737:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12225.706980] LustreError: 370737:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -17 [12225.717293] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.124@tcp [12225.717293] [12531.173885] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12531.192137] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12531.194350] Lustre: Skipped 2 previous similar messages [12531.199666] Lustre: Skipped 2 previous similar messages [12531.206738] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12535.640230] Lustre: server umount lustre-MDT0001 complete [12536.293043] LustreError: 358578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12536.322888] LustreError: 358578:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [12545.550986] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12545.637743] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12545.642432] Lustre: Found index 1 for lustre-MDT0001, updating log [12545.685911] Lustre: 358573:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12546.130425] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12550.811434] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12551.183221] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12554.226267] LustreError: 379079:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12554.233258] LustreError: 379079:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12554.239872] LustreError: 358576:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12554.243878] LustreError: 358576:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12554.315122] LustreError: 379082:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12554.327808] LustreError: 379082:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.124@tcp: cfg command failed: rc = -114 [12554.338252] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12554.338252] [12873.697516] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12873.702721] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12873.709562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12873.713429] Lustre: Skipped 2 previous similar messages [12879.346336] Lustre: server umount lustre-MDT0000 complete [12881.895021] LustreError: 369091:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12881.910884] LustreError: 369091:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [12883.363974] LustreError: 358565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786756760 with bad export cookie 11281661096977408956 [12883.375667] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12883.384863] LustreError: 358565:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12887.011920] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12887.021142] Lustre: Skipped 2 previous similar messages [12890.300668] Lustre: server umount lustre-MDT0001 complete [12894.821978] Lustre: server umount lustre-OST0000 complete [12908.080626] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [12923.421167] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [12966.950429] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 21:20:41 (1786756841) [12976.055536] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12976.345295] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12976.375604] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12976.442431] Lustre: lustre-MDT0000: new disk, initializing [12976.521942] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12976.577641] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12983.426775] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [12995.206682] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12995.341552] Lustre: 389429:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12995.418384] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12995.422607] Lustre: Skipped 1 previous similar message [12995.486044] Lustre: lustre-MDT0001: new disk, initializing [12995.582942] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12995.623373] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12995.639298] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13000.766723] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13010.527793] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13016.803806] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13023.493585] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13023.841536] Lustre: lustre-OST0000: new disk, initializing [13023.844935] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13023.851440] Lustre: 391400:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13023.969548] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13031.763400] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_default_debug -1 all [13032.584906] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13032.605207] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13032.839195] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13042.552957] Lustre: DEBUG MARKER: oleg124-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13057.000700] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13057.015203] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13057.033093] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13057.035514] Lustre: Skipped 5 previous similar messages [13059.457281] Lustre: server umount lustre-MDT0000 complete [13063.660692] LustreError: 390471:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13063.688735] LustreError: 390471:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [13064.240343] LustreError: 391399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786756941 with bad export cookie 11281661096977410909 [13064.246249] LustreError: MGC192.168.201.124@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13064.254191] LustreError: 391399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [13064.577734] Lustre: server umount lustre-MDT0001 complete [13079.390160] Lustre: server umount lustre-OST0000 complete [13115.606789] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing set_hostid [13126.167673] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing load_modules_local [13158.779174] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12810 sec ========= 21:23:53 (1786757033) [13160.569155] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 21:23:55 (1786757035) === [13164.815855] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 21:23:58 (1786757038) === [13208.274184] Lustre: DEBUG MARKER: oleg124-server.virtnet: executing unload_modules_local [13210.679427] Key type lgssc unregistered [13210.865571] LNet: 397768:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13210.870845] LNetError: 397768:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13210.886550] LNet: Removed LNI 192.168.201.124@tcp [13211.684110] Key type .llcrypt unregistered [13211.685926] Key type ._llcrypt unregistered