[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 523535764 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001011] APIC: Switch to symmetric I/O mode setup [ 0.003251] x2apic enabled [ 0.004007] Switched APIC routing to physical x2apic. [ 0.005015] kvm-guest: setup PV IPIs [ 0.008000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008022] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009015] pid_max: default: 32768 minimum: 301 [ 0.011120] LSM: Security Framework initializing [ 0.012055] Yama: becoming mindful. [ 0.013041] SELinux: Initializing. [ 0.014071] *** VALIDATE selinux *** [ 0.023293] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.028411] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.029162] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030118] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031127] *** VALIDATE tmpfs *** [ 0.032472] *** VALIDATE proc *** [ 0.034218] *** VALIDATE cgroup *** [ 0.035010] *** VALIDATE cgroup2 *** [ 0.036261] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037159] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039004] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040028] Spectre V2 : User space: Vulnerable [ 0.041009] Speculative Store Bypass: Vulnerable [ 0.044441] debug: unmapping init [mem 0xffffffff88259000-0xffffffff88260fff] [ 0.046996] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047662] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048026] ... version: 2 [ 0.049013] ... bit width: 48 [ 0.050009] ... generic registers: 4 [ 0.051011] ... value mask: 0000ffffffffffff [ 0.052013] ... max period: 00007fffffffffff [ 0.053016] ... fixed-purpose events: 3 [ 0.054010] ... event mask: 000000070000000f [ 0.055285] rcu: Hierarchical SRCU implementation. [ 0.057477] smp: Bringing up secondary CPUs ... [ 0.058569] x86: Booting SMP configuration: [ 0.059026] .... node #0, CPUs: #1 #2 #3 [ 0.062393] smp: Brought up 1 node, 4 CPUs [ 0.064015] smpboot: Max logical packages: 1 [ 0.065020] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.155024] node 0 deferred pages initialised in 86ms [ 0.158078] devtmpfs: initialized [ 0.159256] x86/mm: Memory block size: 128MB [ 0.162794] gcov: version magic: 0x41383552 [ 0.165333] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.166083] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.167308] pinctrl core: initialized pinctrl subsystem [ 0.168203] [ 0.168726] ************************************************************* [ 0.169011] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.170010] ** ** [ 0.171018] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.172012] ** ** [ 0.173010] ** This means that this kernel is built to expose internal ** [ 0.174014] ** IOMMU data structures, which may compromise security on ** [ 0.175014] ** your system. ** [ 0.176016] ** ** [ 0.177013] ** If you see this message and you are not debugging the ** [ 0.178017] ** kernel, report this immediately to your vendor! ** [ 0.179016] ** ** [ 0.180017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.181016] ************************************************************* [ 0.182727] NET: Registered protocol family 16 [ 0.183476] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.184069] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.185065] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.186545] cpuidle: using governor menu [ 0.188933] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.192603] PCI: Using configuration type 1 for base access [ 0.195152] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.204091] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.205041] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.209015] cryptd: max_cpu_qlen set to 1000 [ 0.211244] ACPI: Added _OSI(Module Device) [ 0.213014] ACPI: Added _OSI(Processor Device) [ 0.215014] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.217014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.222754] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.231162] ACPI: Interpreter enabled [ 0.233072] ACPI: PM: (supports S0 S3 S4 S5) [ 0.234013] ACPI: Using IOAPIC for interrupt routing [ 0.236102] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.241527] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.252408] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.255151] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.258022] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.262081] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.267383] acpiphp: Slot [2] registered [ 0.269243] acpiphp: Slot [5] registered [ 0.271203] acpiphp: Slot [6] registered [ 0.273177] acpiphp: Slot [7] registered [ 0.275403] acpiphp: Slot [8] registered [ 0.278093] acpiphp: Slot [9] registered [ 0.280196] acpiphp: Slot [10] registered [ 0.282171] acpiphp: Slot [3] registered [ 0.283199] acpiphp: Slot [4] registered [ 0.285130] acpiphp: Slot [11] registered [ 0.287135] acpiphp: Slot [12] registered [ 0.288153] acpiphp: Slot [13] registered [ 0.290142] acpiphp: Slot [14] registered [ 0.292129] acpiphp: Slot [15] registered [ 0.294135] acpiphp: Slot [16] registered [ 0.296129] acpiphp: Slot [17] registered [ 0.297115] acpiphp: Slot [18] registered [ 0.299146] acpiphp: Slot [19] registered [ 0.301165] acpiphp: Slot [20] registered [ 0.303114] acpiphp: Slot [21] registered [ 0.304122] acpiphp: Slot [22] registered [ 0.306116] acpiphp: Slot [23] registered [ 0.308134] acpiphp: Slot [24] registered [ 0.310117] acpiphp: Slot [25] registered [ 0.312118] acpiphp: Slot [26] registered [ 0.313119] acpiphp: Slot [27] registered [ 0.315175] acpiphp: Slot [28] registered [ 0.316124] acpiphp: Slot [29] registered [ 0.318118] acpiphp: Slot [30] registered [ 0.319119] acpiphp: Slot [31] registered [ 0.321087] PCI host bridge to bus 0000:00 [ 0.323021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.325024] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.328066] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.330028] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.333022] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.335020] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.337237] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.340051] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.343102] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.354774] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.359062] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.362034] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.364019] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.368019] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.370578] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.373876] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.376041] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.379571] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.385019] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.401017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.406014] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.411743] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.424027] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.437014] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.463025] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.473639] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.482024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.490024] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.510019] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.522044] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.528028] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.534052] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.555024] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.569518] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.575012] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.582018] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.602016] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.611281] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.624018] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.632020] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.656019] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.665195] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.675023] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.683019] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.703019] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.716127] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.718443] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.721556] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.724464] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.727336] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.732080] iommu: Default domain type: Passthrough [ 0.733000] SCSI subsystem initialized [ 0.734293] ACPI: bus type USB registered [ 0.735221] usbcore: registered new interface driver usbfs [ 0.737139] usbcore: registered new interface driver hub [ 0.739135] usbcore: registered new device driver usb [ 0.741268] pps_core: LinuxPPS API ver. 1 registered [ 0.742019] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.744109] PTP clock support registered [ 0.746054] EDAC MC: Ver: 3.0.0 [ 0.747584] PCI: Using ACPI for IRQ routing [ 0.748423] NetLabel: Initializing [ 0.749000] NetLabel: domain hash size = 128 [ 0.752018] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.754089] NetLabel: unlabeled traffic allowed by default [ 0.756357] vgaarb: loaded [ 0.758365] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.760021] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.766685] clocksource: Switched to clocksource kvm-clock [ 0.880316] VFS: Disk quotas dquot_6.6.0 [ 0.882124] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.884467] *** VALIDATE ramfs *** [ 0.885494] *** VALIDATE hugetlbfs *** [ 0.886786] pnp: PnP ACPI init [ 0.888383] pnp: PnP ACPI: found 6 devices [ 0.910174] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 0.914127] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 0.916476] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 0.919019] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 0.921956] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 0.924462] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 0.927884] NET: Registered protocol family 2 [ 0.930574] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 0.935915] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 0.940057] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 0.945374] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 0.949847] TCP: Hash tables configured (established 65536 bind 65536) [ 0.953228] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 0.956843] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.960286] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 0.963781] NET: Registered protocol family 1 [ 0.967059] RPC: Registered named UNIX socket transport module. [ 0.969961] RPC: Registered udp transport module. [ 0.972312] RPC: Registered tcp transport module. [ 0.974511] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.977623] NET: Registered protocol family 44 [ 0.979524] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 0.981422] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 0.982829] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 0.984710] PCI: CLS 0 bytes, default 64 [ 0.986370] Unpacking initramfs... [ 2.456315] debug: unmapping init [mem 0xffff97c67cc54000-0xffff97c67ffbffff] [ 2.460305] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.462696] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.465713] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 2.971091] Initialise system trusted keyrings [ 2.972935] Key type blacklist registered [ 2.974864] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 2.990243] zbud: loaded [ 2.994918] *** VALIDATE nfs *** [ 2.997096] *** VALIDATE nfs4 *** [ 2.999172] pstore: using deflate compression [ 3.002761] Platform Keyring initialized [ 3.113692] NET: Registered protocol family 38 [ 3.116072] Key type asymmetric registered [ 3.117732] Asymmetric key parser 'x509' registered [ 3.119609] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.122605] io scheduler mq-deadline registered [ 3.123886] io scheduler kyber registered [ 3.125382] io scheduler bfq registered [ 3.127253] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.130553] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.133512] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.136747] ACPI: Power Button [PWRF] [ 3.141654] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.148118] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.174333] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.181864] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.203332] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.237533] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.300936] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.325970] Non-volatile memory driver v1.3 [ 3.327850] Linux agpgart interface v0.103 [ 3.368775] virtio_blk virtio1: [vda] 145792 512-byte logical blocks (74.6 MB/71.2 MiB) [ 3.371629] vda: detected capacity change from 0 to 74645504 [ 3.392962] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.396156] vdb: detected capacity change from 0 to 1073741824 [ 3.412660] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.415745] vdc: detected capacity change from 0 to 2621440000 [ 3.434046] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.437449] vdd: detected capacity change from 0 to 2621440000 [ 3.456825] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.460552] vde: detected capacity change from 0 to 4294967296 [ 3.476569] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.479606] vdf: detected capacity change from 0 to 4294967296 [ 3.486821] libphy: Fixed MDIO Bus: probed [ 3.501749] usbcore: registered new interface driver usbserial_generic [ 3.506417] usbserial: USB Serial support registered for generic [ 3.511731] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.521673] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.525296] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.528501] mousedev: PS/2 mouse device common for all mice [ 3.531871] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.536785] rtc_cmos 00:05: RTC can wake from S4 [ 3.540847] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.548254] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.548545] rtc_cmos 00:05: registered as rtc0 [ 3.553249] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.557385] intel_pstate: CPU model not supported [ 3.561739] hid: raw HID events driver (C) Jiri Kosina [ 3.563952] usbcore: registered new interface driver usbhid [ 3.566009] usbhid: USB HID core driver [ 3.567120] drop_monitor: Initializing network drop monitor service [ 3.569526] Initializing XFRM netlink socket [ 3.571855] NET: Registered protocol family 10 [ 3.576342] Segment Routing with IPv6 [ 3.578170] NET: Registered protocol family 17 [ 3.580667] mpls_gso: MPLS GSO support [ 3.588386] RAS: Correctable Errors collector initialized. [ 3.590135] AVX version of gcm_enc/dec engaged. [ 3.591673] AES CTR mode by8 optimization enabled [ 3.706070] sched_clock: Marking stable (3706025238, 0)->(4678186022, -972160784) [ 3.710521] registered taskstats version 1 [ 3.713369] Loading compiled-in X.509 certificates [ 3.715618] zswap: loaded using pool lzo/zbud [ 3.747390] Key type big_key registered [ 3.761839] Key type encrypted registered [ 3.763691] ima: No TPM chip found, activating TPM-bypass! [ 3.765825] ima: Allocated hash algorithm: sha1 [ 3.767858] ima: No architecture policies found [ 3.769932] evm: Initialising EVM extended attributes: [ 3.772111] evm: security.selinux [ 3.773558] evm: security.ima [ 3.774832] evm: security.capability [ 3.776470] evm: HMAC attrs: 0x1 [ 3.779307] rtc_cmos 00:05: setting system clock to 2026-07-16 06:54:51 UTC (1784184891) [ 3.786423] debug: unmapping init [mem 0xffffffff89203000-0xffffffff893fffff] [ 3.789760] debug: unmapping init [mem 0xffffffff87f82000-0xffffffff88258fff] [ 3.800088] Write protecting the kernel read-only data: 28672k [ 3.802976] debug: unmapping init [mem 0xffffffff86603000-0xffffffff867fffff] [ 3.805746] debug: unmapping init [mem 0xffffffff86f14000-0xffffffff86ffffff] [ 3.848778] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.856536] systemd[1]: Detected virtualization kvm. [ 3.857918] systemd[1]: Detected architecture x86-64. [ 3.859384] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.886827] systemd[1]: No hostname configured. [ 3.889410] systemd[1]: Set hostname to . [ 3.891748] random: systemd: uninitialized urandom read (16 bytes read) [ 3.894155] systemd[1]: Initializing machine ID from random generator. [ 3.934387] random: ln: uninitialized urandom read (6 bytes read) [ 4.034704] random: systemd: uninitialized urandom read (16 bytes read) [ 4.037810] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.043250] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ 4.047424] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on Journal Socket. [ OK ] Started Memstrack Anylazing Service. Starting Setup Virtual Console... [ OK ] Reached target Sockets. Starting Apply Kernel Variables... Starting Journal Service... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Swap. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.714240] device-mapper: uevent: version 1.0.3 [ 4.717229] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.446648] virtio_net virtio0 ens2: renamed from eth0 [ 5.473982] random: fast init done [ 5.584467] scsi host0: ata_piix [ 5.605861] scsi host1: ata_piix [ 5.641342] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.644306] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.419290] dracut-initqueue[586]: RTNETLINK answers: File exists [ 10.281729] random: crng init done [ 10.283281] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 10.825397] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped dracut initqueue hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target System Initialization. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Swap. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.105492] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.464972] SELinux: Disabled at runtime. [ 12.531893] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.542926] systemd[1]: Detected virtualization kvm. [ 12.545263] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.083452] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.087173] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.093578] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.100188] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.104361] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.113688] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.121110] systemd[1]: Starting Remount Root and Kernel File Systems... Starting Remount Root and Kernel File Systems... [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Created slice system-getty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting Huge Pages File System... [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Activating swap /dev/disk/by-label/SWAP... [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Kernel Socket. [ OK ] [ 13.230253] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS Stopped target Initrd Root File System. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting POSIX Message Queue File System... Starting Apply Kernel Variables... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Journal Service. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Mounted Huge Pages File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Apply Kernel Variables. [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... Starting Flush Journal to Persistent Storage... [ OK ] Started udev Coldplug all Devices. [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Mounted /mnt. [ 13.679968] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.988435] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.012878] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.125269] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.141360] EDAC sbridge: Ver: 1.1.2 [ 15.999870] Key type dns_resolver registered [ 16.333595] NFS: Registering the id_resolver key type [ 16.335496] Key type id_resolver registered [ 16.337644] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Create Volatile Files and Directories... Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. [ OK ] Reached target Basic System. Starting Login Service... [ OK ] Started D-Bus System Message Bus. Starting Network Manager... Starting Restore /run/initramfs on shutdown... [ OK ] Reached target sshd-keygen.target. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started irqbalance daemon. [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenSSH server daemon... Starting GSSAPI Proxy Daemon... Starting Dynamic System Tuning Daemon... Starting Hostname Service... [ OK ] Started OpenSSH server daemon. [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg460-server login: [ 29.307543] hrtimer: interrupt took 6529392 ns [ 76.373972] libcfs: loading out-of-tree module taints kernel. [ 76.460270] Key type ._llcrypt registered [ 76.462822] Key type .llcrypt registered [ 76.659487] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 98.020935] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 100.554934] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 100.580468] alg: No test for adler32 (adler32-zlib) [ 101.910169] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 102.465724] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 104.152381] Key type lgssc registered [ 107.056270] Lustre: Echo OBD driver; http://www.lustre.org/ [ 124.246497] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 167.969656] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 182.563546] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 182.603686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 183.908378] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 183.940909] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 184.008359] Lustre: lustre-MDT0000: new disk, initializing [ 184.110917] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 184.125161] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 188.565670] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 201.889471] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 201.977403] Lustre: 6511:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 202.001162] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 202.005507] Lustre: Skipped 1 previous similar message [ 202.060980] Lustre: lustre-MDT0001: new disk, initializing [ 202.118084] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 202.132947] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 202.142908] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 206.150744] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 211.268411] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 221.772889] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 222.188762] Lustre: lustre-OST0000: new disk, initializing [ 222.193782] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 222.215729] Lustre: 8450:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 222.325214] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 228.606804] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 229.927769] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 229.934777] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 230.030577] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 244.187632] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 244.493561] Lustre: lustre-OST0001: new disk, initializing [ 244.505322] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 244.525372] Lustre: 9523:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 244.713792] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 251.684921] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 254.034937] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 254.046642] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 254.101094] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 264.900815] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 276.607602] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 284.782175] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing check_logdir /tmp/testlogs/ [ 290.785173] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing yml_node [ 296.313874] Lustre: DEBUG MARKER: Client: 2.17.54.162 [ 299.117256] Lustre: DEBUG MARKER: MDS: 2.17.54.162 [ 302.379807] Lustre: DEBUG MARKER: OSS: 2.17.54.162 [ 304.458216] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Jul 16 02:59:49 EDT 2026 [ 324.753733] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 326.578321] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 345.569345] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 345.590522] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 345.621417] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 349.579894] Lustre: server umount lustre-MDT0000 complete [ 353.253773] LustreError: 6504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784185240 with bad export cookie 8709184303790612567 [ 353.257042] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 353.266037] LustreError: 6504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 356.321078] LustreError: 6517:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 356.321855] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 356.327476] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 356.327485] Lustre: Skipped 4 previous similar messages [ 356.335418] LustreError: 6517:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 356.369112] Lustre: Skipped 4 previous similar messages [ 358.370096] LustreError: 7929:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 358.373122] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 358.387357] LustreError: 7929:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 358.411635] Lustre: Skipped 2 previous similar messages [ 359.691532] Lustre: server umount lustre-MDT0001 complete [ 363.836928] Lustre: server umount lustre-OST0000 complete [ 368.119437] Lustre: server umount lustre-OST0001 complete [ 375.083478] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 385.876884] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 428.192269] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 428.507177] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 428.546718] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 428.640344] Lustre: lustre-MDT0000: new disk, initializing [ 428.796386] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 428.816707] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 434.787109] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 447.031472] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 447.266363] Lustre: 15092:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 447.373125] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 447.382838] Lustre: Skipped 1 previous similar message [ 447.515233] Lustre: lustre-MDT0001: new disk, initializing [ 447.652334] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 447.726934] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 447.747561] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 453.541651] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 464.275538] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 472.104575] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 479.872936] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 480.275585] Lustre: lustre-OST0000: new disk, initializing [ 480.283798] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 480.291508] Lustre: 17063:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 480.423483] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 481.883072] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 481.892553] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 482.029627] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 488.493113] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 499.980870] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 505.877231] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 506.153194] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 511.239734] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 511.517400] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 518.123437] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 518.133593] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 518.141118] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 518.196085] Lustre: Skipped 1 previous similar message [ 523.246055] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 523.264188] Lustre: Skipped 1 previous similar message [ 528.352970] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 528.687291] Lustre: server umount lustre-OST0000 complete [ 535.009789] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 535.014703] LustreError: Skipped 1 previous similar message [ 535.019112] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 535.028654] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 535.032922] Lustre: Skipped 3 previous similar messages [ 540.502366] Lustre: server umount lustre-MDT0000 complete [ 543.718307] LustreError: 15103:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 543.765722] LustreError: 15103:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 545.105707] LustreError: 15889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784185432 with bad export cookie 8709184303790614100 [ 545.108452] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 545.110276] LustreError: 15889:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 545.660773] Lustre: server umount lustre-MDT0001 complete [ 551.443849] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 03:03:57 (1784185437) [ 558.348441] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 558.746878] LustreError: 19328:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 558.824964] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 564.171810] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 564.197497] LustreError: 19329:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 573.409399] LustreError: 19328:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 573.424170] LustreError: 19328:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 574.057804] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 579.764149] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 589.164994] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 597.221159] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 606.872256] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 615.437951] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 625.790842] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 634.964489] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 644.582940] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 644.595295] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 644.613696] Lustre: Skipped 3 previous similar messages [ 644.639549] Lustre: Skipped 4 previous similar messages [ 648.997802] Lustre: server umount lustre-MDT0000 complete [ 649.706190] LustreError: 19328:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 649.737994] LustreError: 19328:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 654.522197] LustreError: 19308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784185542 with bad export cookie 8709184303790614982 [ 654.535161] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 654.539905] LustreError: 19308:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 654.828168] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 655.130637] Lustre: server umount lustre-MDT0001 complete [ 660.739128] Lustre: server umount lustre-OST0000 complete [ 688.593458] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 702.307238] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 702.966766] LustreError: 23716:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 703.003026] LustreError: 23716:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 703.048718] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 703.052503] Lustre: Skipped 2 previous similar messages [ 708.551571] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 720.808238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 727.795131] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 731.972724] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 741.935505] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 750.928204] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 762.148926] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 762.472779] Lustre: lustre-OST0001: new disk, initializing [ 762.487798] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 762.511056] Lustre: 26038:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 763.996061] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 764.006645] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 764.116609] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 770.448866] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 782.425426] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 786.740363] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 803.297930] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 803.304911] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 803.318379] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 808.211339] Lustre: server umount lustre-MDT0000 complete [ 810.470097] LustreError: 23717:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 810.484038] LustreError: 23717:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 812.228734] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784185699 with bad export cookie 8709184303790615829 [ 812.234274] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 812.243612] LustreError: 23696:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 812.620600] Lustre: server umount lustre-MDT0001 complete [ 827.410334] Lustre: server umount lustre-OST0000 complete [ 831.456266] Lustre: 3645:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784185703/real 1784185703] req@ffff97c5cb877800 x1870853560342016/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784185719 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 832.305692] Lustre: server umount lustre-OST0001 complete [ 852.432109] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 867.057864] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 867.687573] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 867.692880] Lustre: Skipped 3 previous similar messages [ 873.801385] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 884.911610] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 890.480122] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 893.950710] Lustre: 30204:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 902.028748] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 909.837675] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 920.500410] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 928.536925] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 938.096922] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 942.506692] Lustre: 32086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 951.782409] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 951.796727] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 951.813208] Lustre: Skipped 6 previous similar messages [ 951.816059] Lustre: Skipped 5 previous similar messages [ 951.835639] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 956.620594] Lustre: server umount lustre-MDT0000 complete [ 956.911409] LustreError: 29057:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 956.929591] LustreError: 29057:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 9 previous similar messages [ 961.831584] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 961.829522] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784185849 with bad export cookie 8709184303790617124 [ 961.880389] LustreError: 29036:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 962.370504] Lustre: server umount lustre-MDT0001 complete [ 970.230058] Lustre: server umount lustre-OST0000 complete [ 974.879935] Lustre: server umount lustre-OST0001 complete [ 986.298660] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 03:11:11 (1784185871) [ 1020.724043] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1035.251975] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1041.808800] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1052.615945] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1058.693514] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1062.311829] Lustre: 36040:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1071.854662] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1083.315773] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1097.239175] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1107.141151] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1116.135083] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1120.732523] Lustre: 37919:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1128.935860] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1128.955122] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1128.967277] Lustre: Skipped 6 previous similar messages [ 1128.997795] Lustre: Skipped 7 previous similar messages [ 1134.286644] Lustre: server umount lustre-MDT0000 complete [ 1138.792246] LustreError: 34871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784186026 with bad export cookie 8709184303790618230 [ 1138.797978] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1138.806739] LustreError: 34871:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1139.445146] Lustre: server umount lustre-MDT0001 complete [ 1146.182850] Lustre: server umount lustre-OST0000 complete [ 1151.194931] Lustre: server umount lustre-OST0001 complete [ 1162.619980] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 03:14:08 (1784186048) [ 1182.023867] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1194.723995] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1195.330199] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1195.337376] Lustre: Skipped 7 previous similar messages [ 1200.949734] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1210.454826] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1216.428701] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1219.933587] Lustre: 41209:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1228.645727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1235.372711] LustreError: 41562:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1235.394278] LustreError: 41562:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 30 previous similar messages [ 1237.028659] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1249.238268] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1258.910773] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1267.741575] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1271.732842] Lustre: 43090:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1272.752841] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1272.758620] Lustre: Skipped 1 previous similar message [ 1277.411284] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1277.419216] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1277.442973] Lustre: Skipped 2 previous similar messages [ 1282.960349] Lustre: server umount lustre-MDT0000 complete [ 1287.560169] LustreError: 40043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784186175 with bad export cookie 8709184303790619336 [ 1287.573726] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1294.081374] Lustre: server umount lustre-MDT0001 complete [ 1304.707900] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1313.252163] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x78dd3cfb26d8ff0c [ 1313.277571] Lustre: MGC192.168.204.160@tcp: Connection restored to 0@lo (at 0@lo) [ 1319.225616] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1322.932449] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1329.317594] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1335.296063] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1335.299835] Lustre: Skipped 1 previous similar message [ 1335.631977] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1345.051238] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1347.365177] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1354.934708] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1357.195434] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1365.999047] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1370.486850] Lustre: server umount lustre-MDT0000 complete [ 1375.102126] LustreError: 40043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784186262 with bad export cookie 8709184303790620428 [ 1375.110836] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1375.115366] LustreError: 40043:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 7 previous similar messages [ 1375.616449] Lustre: server umount lustre-MDT0001 complete [ 1390.461340] Lustre: server umount lustre-OST0000 complete [ 1391.458477] Lustre: 3645:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784186263/real 1784186263] req@ffff97c6fd8e4700 x1870853560551168/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784186279 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1394.649774] Lustre: server umount lustre-OST0001 complete [ 1404.665569] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 03:18:10 (1784186290) [ 1423.642739] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1436.992322] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1443.434302] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1454.666907] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1461.409630] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1465.496184] Lustre: 48854:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1474.333110] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1482.446494] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1493.191949] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1501.187121] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1510.349788] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1515.923483] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1529.328812] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1529.335270] Lustre: Skipped 19 previous similar messages [ 1534.657739] Lustre: server umount lustre-MDT0000 complete [ 1539.238269] LustreError: 47683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784186426 with bad export cookie 8709184303790621254 [ 1539.242315] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1539.247503] LustreError: 47683:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1539.559973] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1539.578724] Lustre: Skipped 14 previous similar messages [ 1539.723903] Lustre: server umount lustre-MDT0001 complete [ 1546.337902] Lustre: server umount lustre-OST0000 complete [ 1550.673188] Lustre: server umount lustre-OST0001 complete [ 1561.863329] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 03:20:47 (1784186447) [ 1564.704817] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1566.318337] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 03:20:52 (1784186452) [ 1577.678203] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 1590.005899] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1614.628062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1614.886269] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1614.942553] Lustre: lustre-MDT0000: new disk, initializing [ 1615.022671] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1620.034667] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1630.133410] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1637.715919] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1637.978796] Lustre: lustre-OST0000: new disk, initializing [ 1637.983588] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1637.987845] Lustre: Skipped 1 previous similar message [ 1637.998566] Lustre: 54935:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1639.630891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1639.642664] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1639.685678] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1644.597916] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1655.538066] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1658.225991] Lustre: DEBUG MARKER: create test files [ 1696.230669] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1698.315087] Lustre: server umount lustre-OST0000 complete [ 1702.420380] Lustre: server umount lustre-MDT0000 complete [ 1712.890906] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 1716.293953] Key type lgssc unregistered [ 1716.583901] LNet: 56566:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1716.592575] LNetError: 56566:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1716.614596] LNet: Removed LNI 192.168.204.160@tcp [ 1717.565178] Key type .llcrypt unregistered [ 1717.568432] Key type ._llcrypt unregistered [ 1749.695535] Key type ._llcrypt registered [ 1749.697788] Key type .llcrypt registered [ 1749.873764] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1751.289612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1751.324236] alg: No test for adler32 (adler32-zlib) [ 1752.564955] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 1752.965787] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 1754.624173] Key type lgssc registered [ 1755.971803] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1765.046844] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1765.079988] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1766.488313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1771.297944] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1778.617452] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1786.324472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1786.633108] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1791.790832] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:98 to 0x240000400:129) [ 1791.886934] Lustre: 59302:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1791.901173] Lustre: 59302:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1791.909753] Lustre: 59302:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1791.918693] Lustre: 59302:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1791.931192] Lustre: 59302:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1791.939969] Lustre: 59302:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1792.669249] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1800.674527] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1805.893495] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1812.258103] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1821.863973] Lustre: server umount lustre-OST0000 complete [ 1837.157842] Lustre: server umount lustre-MDT0000 complete [ 1847.903363] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 1851.164413] Key type lgssc unregistered [ 1851.479897] LNet: 60914:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1851.493977] LNetError: 60914:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1851.527521] LNet: Removed LNI 192.168.204.160@tcp [ 1852.627517] Key type .llcrypt unregistered [ 1852.632944] Key type ._llcrypt unregistered [ 1886.169869] Key type ._llcrypt registered [ 1886.173648] Key type .llcrypt registered [ 1886.296912] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 1887.784732] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1887.849800] alg: No test for adler32 (adler32-zlib) [ 1888.952577] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 1889.156925] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 1890.856212] Key type lgssc registered [ 1892.050995] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1900.368836] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1900.412552] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1901.971127] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1907.190317] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1915.608497] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1923.151154] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1923.667459] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1927.733070] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1927.779572] Lustre: 63748:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1927.799054] Lustre: 63748:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1927.808942] Lustre: 63748:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1927.824062] Lustre: 63748:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1927.833283] Lustre: 63748:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1927.846097] Lustre: 63748:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1929.763907] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 1937.216421] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1943.541906] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1953.254468] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1953.286363] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1958.375196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1958.720965] Lustre: server umount lustre-OST0000 complete [ 1962.903172] Lustre: server umount lustre-MDT0000 complete [ 1975.554487] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 1978.639851] Key type lgssc unregistered [ 1978.969247] LNet: 65309:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1978.974861] LNetError: 65309:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1978.991823] LNet: Removed LNI 192.168.204.160@tcp [ 1979.975557] Key type .llcrypt unregistered [ 1979.978293] Key type ._llcrypt unregistered [ 1997.353762] Key type ._llcrypt registered [ 1997.357442] Key type .llcrypt registered [ 1997.490083] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 2014.048718] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 2014.958716] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2015.045737] alg: No test for adler32 (adler32-zlib) [ 2016.048558] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 2016.337479] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 2018.016126] Key type lgssc registered [ 2019.370947] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2056.010392] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2056.045244] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2057.271372] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2057.302559] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2057.385392] Lustre: lustre-MDT0000: new disk, initializing [ 2057.479953] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2057.500652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2061.635782] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2073.989288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2074.219679] Lustre: 68464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2074.322000] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2074.324871] Lustre: Skipped 1 previous similar message [ 2074.426397] Lustre: lustre-MDT0001: new disk, initializing [ 2074.570764] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2074.640175] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2074.662551] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2079.166794] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2090.996499] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2097.429789] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2104.891485] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2105.163931] Lustre: lustre-OST0000: new disk, initializing [ 2105.167705] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2105.177730] Lustre: 70432:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2105.257473] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2111.580596] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2111.588049] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2111.716146] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2112.413745] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2122.425611] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2126.817649] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2127.015313] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2132.270845] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2132.496646] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2137.574911] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2137.577657] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2137.586289] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2137.606733] Lustre: Skipped 1 previous similar message [ 2140.882547] Lustre: server umount lustre-OST0000 complete [ 2146.278456] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2146.286734] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2146.300515] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2146.308558] Lustre: Skipped 1 previous similar message [ 2147.809899] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2147.811637] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2150.585235] Lustre: server umount lustre-MDT0000 complete [ 2154.034369] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187041 with bad export cookie 2364498602396997083 [ 2154.037139] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2154.043947] LustreError: 68455:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2154.336240] Lustre: server umount lustre-MDT0001 complete [ 2162.659783] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 03:30:48 (1784187048) [ 2174.380472] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 2212.432936] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 2225.157355] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 2262.138138] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2262.432618] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2262.486503] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2262.579637] Lustre: lustre-MDT0000: new disk, initializing [ 2262.664295] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2262.685960] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2266.675995] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2277.363202] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2277.535929] Lustre: 76453:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2277.629316] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2277.643851] Lustre: Skipped 1 previous similar message [ 2277.793799] Lustre: lustre-MDT0001: new disk, initializing [ 2277.912408] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2277.965556] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2277.977872] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2282.464720] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2291.976705] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2297.871467] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2304.687217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2304.992727] Lustre: lustre-OST0000: new disk, initializing [ 2305.002683] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2305.013255] Lustre: 78421:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2305.121154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2310.792895] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2313.797844] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2313.807942] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2313.926853] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2319.673065] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2324.160310] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2324.426582] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2329.241835] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2329.511659] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2334.693084] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2334.695378] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2334.706578] LustreError: Skipped 1 previous similar message [ 2334.725438] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2334.731592] Lustre: Skipped 1 previous similar message [ 2334.753735] Lustre: Skipped 2 previous similar messages [ 2337.404838] Lustre: server umount lustre-OST0000 complete [ 2344.421344] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2344.433966] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2344.450934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2347.398748] Lustre: server umount lustre-MDT0000 complete [ 2350.051357] LustreError: 79395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2350.079454] LustreError: 79395:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2350.887486] LustreError: 76443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187238 with bad export cookie 2364498602396997965 [ 2350.888851] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2350.898665] LustreError: 76443:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 2351.305610] Lustre: server umount lustre-MDT0001 complete [ 2359.702443] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 03:34:05 (1784187245) [ 2367.908249] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2368.388223] LustreError: 80834:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2368.539139] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2373.602559] LustreError: 80835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2375.163775] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2378.724248] LustreError: 80834:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2383.845711] LustreError: 80835:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2385.629582] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2390.714320] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2398.660060] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2404.300860] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2411.346077] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2411.857064] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2411.865058] Lustre: Skipped 1 previous similar message [ 2418.424812] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2425.613555] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2428.971315] Lustre: 83516:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 5e9924aa-638f-4e32-8db2-b0aea49bef69 at adminstrative request [ 2429.417706] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2429.431622] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2439.592199] Lustre: MGS: Client 5e9924aa-638f-4e32-8db2-b0aea49bef69 (at 0@lo) reconnecting [ 2439.622727] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x20d062f383c7dcbf to 0x20d062f383c7df58 [ 2439.639266] Lustre: MGC192.168.204.160@tcp: Connection restored to 0@lo (at 0@lo) [ 2466.490959] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2466.668989] Lustre: lustre-OST0001: new disk, initializing [ 2466.672779] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2466.678459] Lustre: 83911:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2473.496598] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2475.636153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2475.646472] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2475.774222] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2483.372181] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2491.371770] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2491.374463] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2491.377083] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2491.377091] Lustre: Skipped 1 previous similar message [ 2491.388988] Lustre: Skipped 2 previous similar messages [ 2492.655104] Lustre: server umount lustre-OST0001 complete [ 2496.484789] LustreError: 82656:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2496.510106] LustreError: 82656:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2501.091464] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2501.102951] LustreError: Skipped 2 previous similar messages [ 2502.904888] Lustre: server umount lustre-OST0000 complete [ 2511.844112] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2511.847184] Lustre: Skipped 7 previous similar messages [ 2512.885742] Lustre: server umount lustre-MDT0000 complete [ 2516.869524] LustreError: 80816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187404 with bad export cookie 2364498602396999512 [ 2516.878126] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2516.899459] LustreError: 80816:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2517.530298] Lustre: server umount lustre-MDT0001 complete [ 2527.791469] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 03:36:53 (1784187413) [ 2529.527755] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2531.201177] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 03:36:57 (1784187417) [ 2532.726753] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2534.719208] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 03:37:00 (1784187420) [ 2536.253322] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2537.961722] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 03:37:03 (1784187423) [ 2539.433226] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2541.088423] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 03:37:06 (1784187426) [ 2542.674869] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2548.733751] loop: module loaded [ 2548.778190] loop0: detected capacity change from 0 to 4294967296 [ 2548.846593] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2550.578679] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2554.001561] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2558.161748] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2589.778847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2589.870957] LustreError: 88034:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2589.887315] LustreError: 88034:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2589.901725] LustreError: 88034:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2589.909953] LustreError: 88034:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2610.908669] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 2621.543861] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 2656.411630] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2656.706377] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2656.744644] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2656.831361] Lustre: lustre-MDT0000: new disk, initializing [ 2656.935337] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2656.943546] Lustre: Skipped 1 previous similar message [ 2656.955704] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2662.400482] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2672.917039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2673.011094] Lustre: 90900:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2673.320545] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2677.570542] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2687.358205] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2694.683807] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2702.155832] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2702.517407] Lustre: 92868:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2710.398608] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2710.633907] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2710.789369] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2720.541662] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2724.804938] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2725.035972] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2729.848695] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2730.121193] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2736.610143] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2736.612316] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2736.616406] LustreError: Skipped 1 previous similar message [ 2736.618861] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2736.656543] Lustre: Skipped 5 previous similar messages [ 2738.565359] Lustre: server umount lustre-OST0000 complete [ 2748.781759] Lustre: server umount lustre-MDT0000 complete [ 2751.969410] LustreError: 90910:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2751.985302] LustreError: 90910:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2752.872873] LustreError: 90892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187640 with bad export cookie 2364498602397000135 [ 2752.880797] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2752.888708] LustreError: 90892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2753.338495] Lustre: server umount lustre-MDT0001 complete [ 2762.928136] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 03:40:48 (1784187648) [ 2771.089380] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2776.855783] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2785.993166] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2786.348032] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2786.356253] Lustre: Skipped 3 previous similar messages [ 2791.236031] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2799.463944] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2806.540305] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2814.368373] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2821.211828] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2829.430281] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2834.004603] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2834.286878] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2839.207528] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2839.440484] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2847.160295] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2847.438359] Lustre: lustre-OST0001: new disk, initializing [ 2847.441959] Lustre: Skipped 2 previous similar messages [ 2847.446868] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2847.454635] Lustre: Skipped 3 previous similar messages [ 2847.469439] Lustre: 98652:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2854.453954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2854.478177] Lustre: Skipped 2 previous similar messages [ 2854.488765] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2854.612125] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2854.744695] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2865.709320] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2871.086783] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2871.293375] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2876.306551] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2876.547432] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2900.088272] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2930.918274] Lustre: Failing over lustre-MDT0000 [ 2931.154548] Lustre: lustre-MDT0000: Not available for connect from 192.168.204.60@tcp (stopping) [ 2931.168463] Lustre: Skipped 5 previous similar messages [ 2931.311652] LustreError: 100753:0:(ldlm_resource.c:1207:ldlm_resource_complain()) MGS: namespace resource [0x65727473756c:0x0:0x0].0x0 (00000000530e9f7a) refcount nonzero (2) after lock cleanup; forcing cleanup. [ 2931.422924] Lustre: server umount lustre-MDT0000 complete [ 2934.758133] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2934.768940] LustreError: Skipped 1 previous similar message [ 2934.772385] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2934.784528] Lustre: Skipped 2 previous similar messages [ 2934.798233] LustreError: 96009:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2934.816108] LustreError: 96009:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2952.174661] Lustre: 66687:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784187823/real 1784187823] req@0000000007606e54 x1870855567331712/t0(0) o400->MGC192.168.204.160@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1784187839 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2952.227220] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2957.066475] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2957.069467] LDISKFS-fs (dm-0): recovery complete [ 2957.075862] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2961.392136] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x0 to 0x20d062f383c8a9d3 [ 2961.413012] Lustre: MGC192.168.204.160@tcp: Connection restored to 0@lo (at 0@lo) [ 2961.846650] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2966.608650] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 2966.968273] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2966.972841] Lustre: 95277:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2967.044290] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2967.121140] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2967.456253] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2967.798052] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2967.823264] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 8 previous similar messages [ 2967.840592] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2967.855471] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2968.176731] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2968.198164] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 2968.849689] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2968.870173] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 2 previous similar messages [ 2968.877724] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2968.891133] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2969.214595] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2969.227942] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 2 previous similar messages [ 2971.140154] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2971.146110] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 2971.156799] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2971.174430] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2971.504141] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2971.519086] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 2975.345378] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2975.362719] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 2975.380362] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2975.392858] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2975.720185] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2975.737165] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 2983.452989] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2983.467789] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 24 previous similar messages [ 2983.483383] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2983.492971] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2983.817292] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2983.826810] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 24 previous similar messages [ 2999.497992] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2999.525102] Lustre: 101504:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 49 previous similar messages [ 2999.536472] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2999.553554] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 2999.882113] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2999.893339] LustreError: 101504:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 49 previous similar messages [ 3026.058095] Lustre: 101504:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3026.067080] Lustre: 101504:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3026.078178] Lustre: 101504:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 910a53fd-7276-4c6c-a35b-417de0cd58a6@192.168.204.60@tcp [ 3026.102663] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3026.115667] Lustre: 101504:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000084b575d x1870855549294976/t0(8589934780) o35->910a53fd-7276-4c6c-a35b-417de0cd58a6@192.168.204.60@tcp:614/0 lens 392/0 e 0 to 0 dl 1784187924 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3026.138928] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3026.163078] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3026.180241] Lustre: lustre-MDT0000: Denying connection for new client 910a53fd-7276-4c6c-a35b-417de0cd58a6 (at 192.168.204.60@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3026.221540] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3026.224686] Lustre: Skipped 2 previous similar messages [ 3026.235556] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3026.280863] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3026.285820] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3041.252061] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3041.264110] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3041.282477] Lustre: Skipped 3 previous similar messages [ 3047.210267] Lustre: server umount lustre-OST0000 complete [ 3048.931209] LustreError: 97102:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3048.957346] LustreError: 97102:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 41 previous similar messages [ 3057.352517] Lustre: server umount lustre-OST0001 complete [ 3062.247218] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3062.254666] Lustre: Skipped 7 previous similar messages [ 3067.485302] Lustre: server umount lustre-MDT0000 complete [ 3071.490452] LustreError: 95261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784187959 with bad export cookie 2364498602397051347 [ 3071.492240] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3071.501287] LustreError: 95261:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3071.948055] Lustre: server umount lustre-MDT0001 complete [ 3081.505020] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:46:07 (1784187967) [ 3089.674101] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3089.685646] Lustre: *** cfs_fail_loc=197, val=0*** [ 3105.314250] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3110.432524] LustreError: 103480:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.160@tcp: failed processing log, type 4: rc = -110 [ 3115.575161] Lustre: *** cfs_fail_loc=197, val=0*** [ 3115.583102] Lustre: Skipped 65 previous similar messages [ 3136.162273] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3136.173941] Lustre: Skipped 3 previous similar messages [ 3143.494427] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3151.007668] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3152.848144] Lustre: server umount lustre-OST0000 complete [ 3164.174770] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:47:29 (1784188049) [ 3186.412906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3186.926697] LustreError: 105102:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3186.947805] LustreError: 105102:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 3192.793779] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3203.111605] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3207.124452] LustreError: 105133:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3207.135304] LustreError: 105133:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3207.142594] LustreError: 105133:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 20, retries 0, failed: rc = -5 [ 3207.540222] Lustre: server umount lustre-MDT0000 complete [ 3242.892831] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 3254.127155] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 3298.409029] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3298.763320] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3298.793659] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3298.953706] Lustre: lustre-MDT0000: new disk, initializing [ 3299.087621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3304.662295] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3317.644339] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3317.756442] Lustre: 109312:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3318.025251] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3323.439754] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3334.272419] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3340.377615] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3348.327817] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3348.754399] Lustre: 111279:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3355.962635] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3358.413658] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3366.007356] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3371.650981] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3371.900655] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3377.070912] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3377.318363] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3379.169180] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3379.180512] LustreError: Skipped 3 previous similar messages [ 3379.183607] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3379.190707] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3379.201601] Lustre: Skipped 6 previous similar messages [ 3379.215752] Lustre: Skipped 2 previous similar messages [ 3385.380741] Lustre: server umount lustre-OST0000 complete [ 3395.543900] Lustre: server umount lustre-MDT0000 complete [ 3399.746332] LustreError: 109304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188287 with bad export cookie 2364498602397062050 [ 3399.752466] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3399.770522] LustreError: 109304:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3400.117270] Lustre: server umount lustre-MDT0001 complete [ 3409.565668] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:51:35 (1784188295) [ 3419.801640] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 3422.856785] Key type lgssc unregistered [ 3423.172991] LNet: 113768:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3423.179369] LNetError: 113768:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3423.199700] LNet: Removed LNI 192.168.204.160@tcp [ 3424.293715] Key type .llcrypt unregistered [ 3424.296462] Key type ._llcrypt unregistered [ 3445.286668] Key type ._llcrypt registered [ 3445.289961] Key type .llcrypt registered [ 3445.487821] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 3464.072515] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 3465.766708] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3465.793967] alg: No test for adler32 (adler32-zlib) [ 3466.901864] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 3467.187627] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 3468.856836] Key type lgssc registered [ 3470.286548] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3517.407840] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3517.436545] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3518.758328] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3518.799607] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3518.897421] Lustre: lustre-MDT0000: new disk, initializing [ 3519.005392] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3519.030174] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3524.296257] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3536.776941] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3537.143566] Lustre: lustre-OST0000: new disk, initializing [ 3537.155878] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3537.163875] Lustre: Skipped 1 previous similar message [ 3537.170724] Lustre: 117945:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3537.292830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3545.248462] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3546.242300] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3546.254595] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3546.318976] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3557.933307] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3558.093698] Lustre: lustre-OST0001: new disk, initializing [ 3558.097909] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3558.104360] Lustre: 118808:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3558.174540] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3565.707990] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3565.727662] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3565.780293] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3565.826984] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3575.781049] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3575.783181] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3575.797547] Lustre: Skipped 1 previous similar message [ 3580.898044] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3580.908104] Lustre: Skipped 2 previous similar messages [ 3582.061667] Lustre: server umount lustre-MDT0000 complete [ 3590.435829] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3595.897312] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188483 with bad export cookie 11825043235246593028 [ 3595.904164] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3595.913953] LustreError: 117132:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3596.073958] Lustre: server umount lustre-OST0000 complete [ 3600.128168] Lustre: server umount lustre-OST0001 complete [ 3609.767069] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 03:54:55 (1784188495) [ 3636.787965] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 03:55:21 (1784188521) [ 3652.010594] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 3680.288811] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:56:05 (1784188565) [ 3727.713491] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 3740.236898] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 3788.012174] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3788.481428] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3788.544508] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3788.675729] Lustre: lustre-MDT0000: new disk, initializing [ 3788.784760] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3788.804198] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3795.258232] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3806.950402] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3807.072736] Lustre: 126593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3807.133945] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3807.141777] Lustre: Skipped 1 previous similar message [ 3807.232724] Lustre: lustre-MDT0001: new disk, initializing [ 3807.347499] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3807.387206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3807.410644] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3812.752615] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3823.415951] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3829.155216] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3837.582721] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3837.958531] Lustre: lustre-OST0000: new disk, initializing [ 3837.962459] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3837.969418] Lustre: 128563:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3838.055692] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3845.069103] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3847.273049] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3847.291959] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3847.423733] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3855.196458] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3860.830528] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3861.175730] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3866.848462] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3867.151691] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3873.251471] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3873.256777] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3873.284515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3873.301582] Lustre: Skipped 1 previous similar message [ 3876.017159] Lustre: server umount lustre-OST0000 complete [ 3883.497237] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3883.504041] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3883.529268] Lustre: Skipped 2 previous similar messages [ 3883.553253] Lustre: Skipped 1 previous similar message [ 3887.072388] Lustre: server umount lustre-MDT0000 complete [ 3891.191741] LustreError: 126586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188778 with bad export cookie 11825043235246594253 [ 3891.211148] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3891.221524] LustreError: 126586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 3891.574942] Lustre: server umount lustre-MDT0001 complete [ 3901.517874] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:59:47 (1784188787) [ 3909.497779] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3910.207548] LustreError: 130974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3910.244481] LustreError: 130974:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3910.307455] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3915.116856] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3915.745760] LustreError: 130975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3919.847249] LustreError: 130974:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3924.972513] LustreError: 130975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3925.204262] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3930.430119] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3940.073347] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3946.648194] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3954.586557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3955.089094] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3955.096516] Lustre: Skipped 1 previous similar message [ 3962.877719] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 3971.293547] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3989.531758] Lustre: *** cfs_fail_loc=159, val=0*** [ 4004.891272] Lustre: lustre-MDT0000: Client f221d4d0-39a2-42d9-9ed0-325c2db34b7b (at 192.168.204.60@tcp) reconnecting [ 4006.736887] Lustre: *** cfs_fail_loc=159, val=0*** [ 4006.745437] Lustre: Skipped 6 previous similar messages [ 4022.261315] Lustre: lustre-MDT0000: Client f221d4d0-39a2-42d9-9ed0-325c2db34b7b (at 192.168.204.60@tcp) reconnecting [ 4027.874461] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4027.889976] LustreError: Skipped 1 previous similar message [ 4027.895900] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4027.927796] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4029.348567] Lustre: server umount lustre-OST0000 complete [ 4036.066126] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4036.076582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4036.078547] Lustre: Skipped 1 previous similar message [ 4036.090395] Lustre: Skipped 1 previous similar message [ 4044.529116] Lustre: server umount lustre-MDT0000 complete [ 4045.290293] LustreError: 133542:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4045.314944] LustreError: 133542:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4048.246100] LustreError: 130954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784188935 with bad export cookie 11825043235246595135 [ 4048.266860] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4048.267514] LustreError: 130954:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4048.763152] Lustre: server umount lustre-MDT0001 complete [ 4060.851268] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 4064.040725] Key type lgssc unregistered [ 4064.353461] LNet: 134781:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4064.368719] LNetError: 134781:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4064.389473] LNet: Removed LNI 192.168.204.160@tcp [ 4065.406757] Key type .llcrypt unregistered [ 4065.409122] Key type ._llcrypt unregistered [ 4094.178872] Key type ._llcrypt registered [ 4094.181180] Key type .llcrypt registered [ 4094.326447] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 4095.414584] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4095.467603] alg: No test for adler32 (adler32-zlib) [ 4096.579654] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 4097.011710] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 4098.824356] Key type lgssc registered [ 4100.321288] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4108.903445] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4108.923235] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4110.448483] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4115.895247] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4120.550431] LustreError: 136197:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4120.567715] LustreError: 136197:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4124.747596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4125.194932] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4129.834854] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4138.726553] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4145.105243] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4153.717214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4154.354481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4155.395167] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4160.978520] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4168.560718] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4177.369603] Lustre: *** cfs_fail_loc=159, val=0*** [ 4193.792684] Lustre: lustre-MDT0000: Client 1b7af689-2a15-4a04-8e70-0e4d62f53905 (at 192.168.204.60@tcp) reconnecting [ 4196.419837] Lustre: *** cfs_fail_loc=159, val=0*** [ 4212.735804] Lustre: lustre-MDT0001: Client 1b7af689-2a15-4a04-8e70-0e4d62f53905 (at 192.168.204.60@tcp) reconnecting [ 4214.608355] Lustre: *** cfs_fail_loc=159, val=0*** [ 4214.614784] Lustre: Skipped 3 previous similar messages [ 4230.178111] Lustre: lustre-MDT0001: Client 1b7af689-2a15-4a04-8e70-0e4d62f53905 (at 192.168.204.60@tcp) reconnecting [ 4236.231620] Lustre: 139230:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4239.643251] Lustre: *** cfs_fail_loc=159, val=0*** [ 4239.648352] Lustre: Skipped 4 previous similar messages [ 4255.233231] Lustre: lustre-MDT0000: Client 0306c883-ea8e-4fe0-a902-f168767a8c12 (at 192.168.204.60@tcp) reconnecting [ 4257.479444] Lustre: *** cfs_fail_loc=159, val=0*** [ 4257.484935] Lustre: Skipped 14 previous similar messages [ 4273.118810] Lustre: lustre-MDT0000: Client 0306c883-ea8e-4fe0-a902-f168767a8c12 (at 192.168.204.60@tcp) reconnecting [ 4274.971419] Lustre: 139521:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4280.810326] LustreError: lustre-OST0000-osc-MDT0001: operation ost_destroy to node 0@lo failed: rc = -107 [ 4280.818683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4280.830965] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4282.346357] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4282.353095] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4286.575941] Lustre: server umount lustre-OST0000 complete [ 4292.593238] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4292.600328] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4292.613911] Lustre: Skipped 1 previous similar message [ 4292.627308] Lustre: Skipped 2 previous similar messages [ 4297.439520] Lustre: server umount lustre-MDT0000 complete [ 4297.698868] LustreError: 136198:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4297.732324] LustreError: 136198:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4301.421018] LustreError: 136180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189189 with bad export cookie 11972924162255671175 [ 4301.421454] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4301.431544] LustreError: 136180:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4301.928265] Lustre: server umount lustre-MDT0001 complete [ 4315.304678] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 4318.528124] Key type lgssc unregistered [ 4318.903913] LNet: 140573:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4318.925620] LNetError: 140573:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4318.965233] LNet: Removed LNI 192.168.204.160@tcp [ 4319.991545] Key type .llcrypt unregistered [ 4319.993916] Key type ._llcrypt unregistered [ 4345.451633] Key type ._llcrypt registered [ 4345.455846] Key type .llcrypt registered [ 4345.579174] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 4346.901580] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4346.941824] alg: No test for adler32 (adler32-zlib) [ 4348.142089] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 4348.396912] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 4350.048274] Key type lgssc registered [ 4351.281223] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4360.649974] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4360.666923] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4362.397408] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4368.387674] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4372.979198] LustreError: 141988:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4373.010193] LustreError: 141988:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4378.086041] LustreError: 141989:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4379.625391] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4380.193474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4385.096704] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4393.864400] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4400.412196] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4407.250471] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4407.602960] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4408.643078] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4414.592575] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4422.636496] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4425.035737] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4431.951523] Lustre: 144575:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4436.228342] Lustre: 144624:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4438.129460] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4439.190878] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4443.561769] Lustre: 144770:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4444.641642] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4445.788586] Lustre: 144868:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4450.789623] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4450.794319] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4450.796232] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4450.808583] Lustre: Skipped 1 previous similar message [ 4450.829024] Lustre: Skipped 1 previous similar message [ 4455.906869] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4455.918571] Lustre: Skipped 1 previous similar message [ 4456.977974] Lustre: server umount lustre-OST0000 complete [ 4461.033103] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4461.041577] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4461.048918] Lustre: Skipped 1 previous similar message [ 4461.059807] Lustre: Skipped 1 previous similar message [ 4466.146381] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4466.158734] Lustre: Skipped 1 previous similar message [ 4467.036649] Lustre: server umount lustre-MDT0000 complete [ 4471.267655] LustreError: 141989:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4471.317311] LustreError: 141989:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4471.763477] LustreError: 141970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189359 with bad export cookie 5598524963231691216 [ 4471.772439] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4471.778789] LustreError: 141970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4472.225492] Lustre: server umount lustre-MDT0001 complete [ 4485.615477] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 4488.759911] Key type lgssc unregistered [ 4489.136934] LNet: 145930:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4489.149806] LNetError: 145930:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4489.181726] LNet: Removed LNI 192.168.204.160@tcp [ 4490.220966] Key type .llcrypt unregistered [ 4490.225299] Key type ._llcrypt unregistered [ 4523.875341] Key type ._llcrypt registered [ 4523.884585] Key type .llcrypt registered [ 4524.068764] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 4525.375532] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4525.472483] alg: No test for adler32 (adler32-zlib) [ 4526.618778] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 4526.933042] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 4528.617974] Key type lgssc registered [ 4529.737181] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4539.409677] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4539.428275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4540.989396] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4546.490475] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4551.141110] LustreError: 147350:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4551.181762] LustreError: 147350:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4555.748865] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4556.175683] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4560.795044] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4569.183923] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4575.866931] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4583.070185] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4583.580209] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4585.661078] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4591.001660] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4599.688949] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4601.029453] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4605.820180] Lustre: *** cfs_fail_loc=159, val=0*** [ 4622.262863] Lustre: lustre-MDT0000: Client 62ecaeeb-57d6-4e76-8a08-1626064a3ff0 (at 192.168.204.60@tcp) reconnecting [ 4632.034558] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4632.050418] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4632.099801] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4632.117191] Lustre: Skipped 1 previous similar message [ 4635.557339] Lustre: server umount lustre-OST0000 complete [ 4642.276046] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4642.283863] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4642.292441] Lustre: Skipped 2 previous similar messages [ 4642.311346] Lustre: Skipped 1 previous similar message [ 4646.737724] Lustre: server umount lustre-MDT0000 complete [ 4647.394790] LustreError: 147350:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4647.421258] LustreError: 147350:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4651.491911] LustreError: 148370:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4651.527067] LustreError: 148370:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4651.704392] LustreError: 148091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189539 with bad export cookie 11057446759436662838 [ 4651.708160] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4651.718803] LustreError: 148091:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4652.243838] Lustre: server umount lustre-MDT0001 complete [ 4666.343607] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 4669.760790] Key type lgssc unregistered [ 4670.238797] LNet: 151017:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4670.258936] LNetError: 151017:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4670.282096] LNet: Removed LNI 192.168.204.160@tcp [ 4671.748281] Key type .llcrypt unregistered [ 4671.755051] Key type ._llcrypt unregistered [ 4700.880416] Key type ._llcrypt registered [ 4700.886101] Key type .llcrypt registered [ 4701.081924] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 4702.363212] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4702.402304] alg: No test for adler32 (adler32-zlib) [ 4703.653366] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 4704.025740] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 4705.816172] Key type lgssc registered [ 4707.501442] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4719.091169] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4719.165566] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4720.868453] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4726.444589] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4730.337488] LustreError: 152431:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4730.354531] LustreError: 152431:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4735.468178] LustreError: 152432:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4737.736407] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4738.400325] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4744.546285] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4754.209751] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4762.123466] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4770.686433] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4771.182731] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4779.303437] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 4782.453250] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4789.031087] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4790.662326] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4811.195801] Lustre: lustre-MDT0001: evicting bf6fcaa6-b6e3-4fe4-9d70-25bcd8abbbdf (at 192.168.204.60@tcp) by administrative request [ 4811.211906] Lustre: Skipped 1 previous similar message [ 4841.958711] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4841.972913] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4842.007587] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4847.106112] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4847.115332] Lustre: Skipped 2 previous similar messages [ 4852.194403] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4852.208414] Lustre: Skipped 1 previous similar message [ 4855.264665] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4855.486212] Lustre: server umount lustre-OST0000 complete [ 4861.418394] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4861.424678] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4861.444703] Lustre: Skipped 1 previous similar message [ 4861.452864] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4862.433751] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4866.827339] Lustre: server umount lustre-MDT0000 complete [ 4867.558803] LustreError: 153163:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4867.580816] LustreError: 153163:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4871.969720] LustreError: 152414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784189759 with bad export cookie 8654137492882263514 [ 4871.980680] LustreError: 152414:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4871.984348] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4872.446500] Lustre: server umount lustre-MDT0001 complete [ 4885.398915] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 4888.486119] Key type lgssc unregistered [ 4888.869975] LNet: 156511:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4888.889020] LNetError: 156511:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4888.909691] LNet: Removed LNI 192.168.204.160@tcp [ 4890.157279] Key type .llcrypt unregistered [ 4890.162888] Key type ._llcrypt unregistered [ 4987.689560] Key type ._llcrypt registered [ 4987.692507] Key type .llcrypt registered [ 4987.889283] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 4989.482047] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4989.591448] alg: No test for adler32 (adler32-zlib) [ 4990.818932] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 4991.307327] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 4993.081616] Key type lgssc registered [ 4994.958294] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5007.872285] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5007.944233] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5009.768466] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5016.101911] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5020.130659] LustreError: 159005:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5020.196245] LustreError: 159005:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5025.250183] LustreError: 159006:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5029.746759] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5030.652701] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5037.526937] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5046.901282] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5053.361205] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5060.912704] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5061.376667] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5068.373737] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5075.696172] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5077.051632] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5079.796840] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5091.297913] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5091.310772] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5091.329601] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5095.397178] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5095.404862] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5095.435994] Lustre: Skipped 1 previous similar message [ 5097.655675] Lustre: server umount lustre-OST0000 complete [ 5105.636977] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5105.641986] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5105.661387] Lustre: Skipped 1 previous similar message [ 5105.681694] Lustre: Skipped 1 previous similar message [ 5108.956165] Lustre: server umount lustre-MDT0000 complete [ 5110.754350] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5110.768310] LustreError: 159001:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5113.550695] LustreError: 158986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190001 with bad export cookie 5231468320062284931 [ 5113.560391] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5113.579431] LustreError: 158986:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5114.188794] Lustre: server umount lustre-MDT0001 complete [ 5127.446631] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 5130.402707] Key type lgssc unregistered [ 5130.677511] LNet: 162567:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5130.691518] LNetError: 162567:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5130.707164] LNet: Removed LNI 192.168.204.160@tcp [ 5131.833949] Key type .llcrypt unregistered [ 5131.835826] Key type ._llcrypt unregistered [ 5162.157685] Key type ._llcrypt registered [ 5162.160545] Key type .llcrypt registered [ 5162.257144] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 5163.414405] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5163.430697] alg: No test for adler32 (adler32-zlib) [ 5164.675287] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 5165.047587] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 5166.712279] Key type lgssc registered [ 5168.202913] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5187.227849] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 04:21:12 (1784190072) [ 5205.653921] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 5217.214845] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 5258.480549] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5258.515080] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5259.749238] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5259.777022] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5259.869886] Lustre: lustre-MDT0000: new disk, initializing [ 5259.963661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5259.984211] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5264.822209] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5276.808296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5276.960739] Lustre: 166836:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5277.019181] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5277.022732] Lustre: Skipped 1 previous similar message [ 5277.096565] Lustre: lustre-MDT0001: new disk, initializing [ 5277.201418] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5277.245905] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5277.260304] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5282.003217] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5292.212689] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5298.615440] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5306.704821] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5307.151247] Lustre: lustre-OST0000: new disk, initializing [ 5307.162545] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5307.171991] Lustre: 168807:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5307.280213] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5313.896375] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5314.648979] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5314.662937] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5314.865589] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5324.628211] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5340.646115] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5340.646966] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5340.652683] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5340.659055] LustreError: Skipped 1 previous similar message [ 5340.689020] Lustre: Skipped 1 previous similar message [ 5342.891504] Lustre: server umount lustre-OST0000 complete [ 5348.835206] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5348.841670] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5348.859968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5348.875914] Lustre: Skipped 1 previous similar message [ 5350.884321] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5350.897687] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5350.918454] Lustre: Skipped 1 previous similar message [ 5353.067858] Lustre: server umount lustre-MDT0000 complete [ 5356.992756] LustreError: 166828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784190244 with bad export cookie 10621280687586800489 [ 5356.995214] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5357.003165] LustreError: 166828:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5357.427581] Lustre: server umount lustre-MDT0001 complete [ 5369.498800] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 5372.513352] Key type lgssc unregistered [ 5372.867532] LNet: 170712:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5372.874044] LNetError: 170712:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5372.900430] LNet: Removed LNI 192.168.204.160@tcp [ 5373.733576] Key type .llcrypt unregistered [ 5373.736527] Key type ._llcrypt unregistered [ 5404.241806] Key type ._llcrypt registered [ 5404.246219] Key type .llcrypt registered [ 5404.389728] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 5405.980430] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5406.044432] alg: No test for adler32 (adler32-zlib) [ 5407.074608] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 5407.450679] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 5409.176290] Key type lgssc registered [ 5410.454953] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5419.672518] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5419.708232] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5421.258565] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5426.407145] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5430.756301] LustreError: 172127:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5430.784159] LustreError: 172127:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5435.490111] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5436.012710] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5440.773914] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5449.444164] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5455.284857] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5462.411557] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5462.719658] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5465.855296] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5469.054350] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 5477.212645] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5488.046055] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5488.072116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5488.091881] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5488.098477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.112963] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5488.896619] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5488.910963] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5488.926908] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5488.935741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.956870] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5490.066668] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5490.081382] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5490.088875] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5490.098461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.113942] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5490.965624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.215553] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5492.222662] Lustre: Skipped 1 previous similar message [ 5492.235834] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5492.250274] Lustre: Skipped 1 previous similar message [ 5492.254109] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5492.263166] Lustre: Skipped 1 previous similar message [ 5492.274461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.288242] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5492.305166] Lustre: Skipped 1 previous similar message [ 5493.249667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.068673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.067927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.108743] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.008977] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5497.014741] Lustre: Skipped 4 previous similar messages [ 5497.022567] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5497.030417] Lustre: Skipped 4 previous similar messages [ 5497.033079] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5497.037514] Lustre: Skipped 4 previous similar messages [ 5497.041292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.050613] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5497.058035] Lustre: Skipped 4 previous similar messages [ 5498.174874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.350578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.412619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.269986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.406192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.472582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.353610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.260474] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5505.267647] Lustre: Skipped 7 previous similar messages [ 5505.288404] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5505.297768] Lustre: Skipped 7 previous similar messages [ 5505.302637] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5505.308433] Lustre: Skipped 7 previous similar messages [ 5505.312811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.329041] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5505.332013] Lustre: Skipped 7 previous similar messages [ 5506.831412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.187412] LustreError: 172146:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5508.203528] LustreError: 172146:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5508.213132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.326121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.608495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.768548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.873570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.791800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.609436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.421752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.207665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.060942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.995464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.929622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.757730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.854623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.600511] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5521.609925] Lustre: Skipped 15 previous similar messages [ 5521.632888] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5521.645033] Lustre: Skipped 15 previous similar messages [ 5521.647429] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5521.652228] Lustre: Skipped 15 previous similar messages [ 5521.655043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.663923] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5521.668494] Lustre: Skipped 15 previous similar messages [ 5522.538060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.394573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.195383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.992750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.753870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.532349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.265573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.997181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.933787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.723405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.730416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.609073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.604899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.601579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.632570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.447234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.358712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.256980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.174620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.184932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.155688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.090935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.978060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.985586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.934913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.972968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.075241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.154760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.031600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.067366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.965417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.977482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.033804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.993269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.830574] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5553.835587] Lustre: Skipped 34 previous similar messages [ 5553.851625] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.860574] Lustre: Skipped 34 previous similar messages [ 5553.863849] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5553.869080] Lustre: Skipped 34 previous similar messages [ 5553.871835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.894633] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5553.904767] Lustre: Skipped 34 previous similar messages [ 5554.904125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.965918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.908417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.936750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.012057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.932680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.933470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.843898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.796674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.018477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.972312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.926397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.823768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.743395] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.901819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.898073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.999914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.371797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.600505] LustreError: 172146:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5573.614106] LustreError: 172146:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5573.657756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.732073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.109039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.269956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.327017] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.540806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.739679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.811252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.986545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.026061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.202812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5586.149217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.354415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.404509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.349352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.219517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.155413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.243708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.395978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.732461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.630562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.749526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.846481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.035303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.089850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.276432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.206198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.282823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.321919] LustreError: 180690:0:(import.c:717:ptlrpc_connect_import_locked()) already connecting [ 5604.324294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.451248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.532454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.505138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.715448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.014770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.266839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.426935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.471131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.847365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.923242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.988268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.929792] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5617.934506] Lustre: Skipped 58 previous similar messages [ 5617.956610] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5617.968511] Lustre: Skipped 58 previous similar messages [ 5617.977116] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5617.986965] Lustre: Skipped 58 previous similar messages [ 5617.992334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.003495] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5618.023521] Lustre: Skipped 58 previous similar messages [ 5619.033094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.257389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.525610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.708223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.635943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.587393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.660583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.591044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.713071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.716479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.573958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.580527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.581692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.580471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.577417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.721134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.862716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.030838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.959876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.193277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.305362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.443598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.448795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.450647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.331241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.318484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.170205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.129282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.935328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.897649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.827085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.782714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.761399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.701581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.735847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.790403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.696704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.744821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.795689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.808362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.890671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.166330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.147548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.362732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.454604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.525746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.758203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.025593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.064433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.264682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.246210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.348045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.487589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.684910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.076531] LustreError: 172146:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5676.094266] LustreError: 172146:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5676.110217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.463244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.593113] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.725796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.638426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.805578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.949683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.796034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.877213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.008134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.232801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.201461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.359616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.342884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.440068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.347371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.397285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.318068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.336805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.229211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.226551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.267527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.387722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.360973] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.215611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.076662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.956180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.922136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.889754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.903551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.923380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.766256] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.747868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.723116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.813532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.805490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.778450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.820592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.810465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.908330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.711780] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.685096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.533871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.500654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.401382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.326845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.241542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.116744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.971840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.055753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.167165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.242711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.401515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.306897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.315843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.296133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.275201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.211623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.096234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.024161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.857456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.640128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.686309] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.650547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.671078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.497591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.457550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.365356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.449873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.417172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.447975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.369180] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5746.377194] Lustre: Skipped 125 previous similar messages [ 5746.402710] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5746.422297] Lustre: Skipped 125 previous similar messages [ 5746.429164] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5746.432570] Lustre: Skipped 125 previous similar messages [ 5746.441473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.450805] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5746.459286] Lustre: Skipped 125 previous similar messages [ 5747.339830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.233440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.160469] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.122871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.232804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.053544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.121394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.103408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.069230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.096347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.013174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.050244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.045687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.039594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.889368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.911830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.843078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.790434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.680760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.621849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.587074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.418512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.342516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.265295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.113504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.133498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.069255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.118824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.988224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.935938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.859609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.875625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.810918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.945311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.068826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.106118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.064559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.217535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.090170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.098872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.047976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.140314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.152039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.093789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.293659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.335275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.396944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.458542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.288375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.253960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.187808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.139861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.117523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.299243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.274887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.370553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.365903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.684910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.819515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.042305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.273930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.667290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.651651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.822549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.762588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.697974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.665407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.692055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.643300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.596374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.446938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.294463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.321630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.218415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.022112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.024573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.775870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.699041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.681560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.777385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.766556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.723584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.718302] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.806171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.907372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.862607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.918230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.939652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.009060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.967479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.843350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.731509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.611598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.640657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.582818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.715688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.690225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.633907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.682708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.796771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.704058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.656925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.832672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.859581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.786682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.622389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.509152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.801233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.629577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.605908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.363680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.341691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.207321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.317500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.360372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.226841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.064343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.086647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.949441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.908613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.836365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.862316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.906130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.882554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.829293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.887284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.926715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.980900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.205091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.122181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.253464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.172806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.311398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.416394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.560950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.469863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.484232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.530076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.573924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.498343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.410291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.299842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.319199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.359675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.422570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.342319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.557817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.553090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.648055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.749622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.099727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.346050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.203201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.153511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.103545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.074083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.172873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.109122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.268996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.261920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.381749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.447264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.744586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.964951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.174584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.385128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.429924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.406422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.738119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.007201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.072429] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.310513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.774225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.112702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.295816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.556604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.655039] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.863987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.125120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.289281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.506266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.379227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.287292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.282767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.236584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.151591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.274518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.519067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.621442] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.357882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.299339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.373678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.294830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.510727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.612169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.731207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.751375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.942940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.937049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.999789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.882807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.963423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.901476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.928994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5956.120245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.079654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.084906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5958.763703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.734521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.557763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.496858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.552164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.589143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5964.636110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.496556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.566106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.556123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.709861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.630363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.560544] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5971.466645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.704816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.842863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.876938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.953552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.150344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.057581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5979.304645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.244530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.297815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.178847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.124202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.152799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.262055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.237503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.225854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.010048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.005381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.222310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.289834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5992.614770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.964188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.148621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.491968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.619801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.500231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.710476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.760539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.689479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.655785] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6002.661467] Lustre: Skipped 249 previous similar messages [ 6002.674355] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6002.684959] Lustre: Skipped 249 previous similar messages [ 6002.693480] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6002.698257] Lustre: Skipped 249 previous similar messages [ 6002.701725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.714513] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6002.722205] Lustre: Skipped 249 previous similar messages [ 6003.861650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.068785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.452963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.704963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.088212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.376593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6011.637893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.703263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.588204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.593249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.581141] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.845355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.994042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.191972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.199776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.162873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.355657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.543741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.597775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.610345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.704812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6027.795650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.851548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.843884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.821312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.832971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.859398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.785517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.731815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.810266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6036.715079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6037.722099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.566086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.384828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.189351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.091893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.012424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6042.914363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.920472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.843901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.602828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.501224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.348318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.422984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.330832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.455461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.486108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.561206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.524595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.824264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.671457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.853362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.070045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.957607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.246980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.807548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.256080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.658745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.942944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.910356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.958935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.068483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.998820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.838932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.844653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.764203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.761580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.811192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.900961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.064611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.091886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.195821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.718811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.815513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.915240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.934568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.095434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.130044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.196704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.218519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.381563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.595320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.967866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.197778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.461767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.584366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.854905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.764495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.934367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.208596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.352818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.614958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.852519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.957762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.870720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.993229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.062875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.045732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.024337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.185813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.140250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.168115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.406259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.574613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.618915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.654538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.649289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.616277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.720045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.938554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.008785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.152320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.182433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.191854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6127.362269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.526657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.575518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.763868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.666634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.653559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.609980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.708417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.573450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.717838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.582069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.616382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.543334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.429514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.344361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.457764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.366213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.273351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.232908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.224732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.049396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.964315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.108928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.359619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.339305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.561949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.782748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.832585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.985953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.128582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.233653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.261688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.248673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.311787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.343354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.344068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.303940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.263575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.457509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.493610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.344656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.259666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6170.494091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.466877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.309561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.172652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6174.054388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.051288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.937301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.890773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6177.839157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.897325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.947703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.892062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6181.696683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.681056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.660295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.722803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.679110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.635941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.556942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6188.630861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.634180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.633201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.482073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.466093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.563619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.556217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.545716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.507513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.567862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.582092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.581041] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.580285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.560029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6202.530246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.566605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.423514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.365803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.194859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.284673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.459688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.667106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.968409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.953733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.896266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.988747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.804763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.886547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.760268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.704864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.638037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.541915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.401559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.474539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.485896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.520603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.544966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.400792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.335232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.389879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.320430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6229.291222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.156645] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.162806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.119359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.027125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.095137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.046968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.167874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.191817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.217698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.286194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6240.284214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.337976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.369305] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.512571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.460388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.471875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6246.384658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.425855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6248.283280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.249372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.464047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.432481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.384152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6253.449561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.556482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.703157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.616758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.701900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.787264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6259.963154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.887599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.875078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.853716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6263.785676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.758617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.744768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.674637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.882078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.844514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.985588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.913167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.807942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.591221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.568639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6274.485109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.352749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.148533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.308077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.346996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.383070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.342822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.272246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.142421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.074227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.012841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.770542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.561718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.417217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.326054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.160888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.034406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.846531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6290.823054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.610400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.517804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.512489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.571907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.560668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.705648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.831509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.834519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.830782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.802253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.762129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.901481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.957769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.928791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.935570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.117616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6308.105658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.159135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.025625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.038054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.983315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.179889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6314.265928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.326852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.405514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.684910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6318.868404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.437475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.973252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6323.262522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.389773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.694660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.950827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.504878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6330.109102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.610610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.970160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.127841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.961696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.094681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.261171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6338.370980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.529576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.603351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.331262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.199434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.310575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6344.450940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.668590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.818678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.838627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6348.929424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.076824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.051411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.852446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.635128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6353.723838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.665799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.772715] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.046765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.818884] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.871773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.862819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.979371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.116871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.204778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.419210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.467731] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6366.326656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.299829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.200569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.207574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.241869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.095788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.971919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.831035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6373.867297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.843152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.857987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.714125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.539089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.499483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.437620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.558749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.356050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6382.205122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.116440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.439480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.659581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.809223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.086590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.548327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.930359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.394076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.753542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.137777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6396.452580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.926245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.007523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.210546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.239458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.316801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.281792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.376344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.126031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.090107] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.919282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.918945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.910051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.900777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6410.783654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.711503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.735076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.798677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.790859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.932451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.006947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.125623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6419.175669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.184231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.389981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.442677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.434622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.413885] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.193343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.752976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.648921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.730643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6430.967542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.865229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.154542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.244327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.530316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.761770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.784889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.092988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6440.297525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.552792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.830471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.020394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.196358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.182549] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.487332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.992139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.105761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.271638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.335338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6453.528464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.879217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.107757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.117641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.282124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.693350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.918960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.020988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.184848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.202108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.390785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.569962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6467.757170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.266647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.503819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.726084] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.810609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.993767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.049745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6476.237708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.387578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.296390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.520903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.651525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6482.015129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.042696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.254749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.643821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.275303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6488.732837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.808812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.831026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6491.851792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.058685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.320179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6495.442760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.419752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.381681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.486949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.670709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6500.635810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.794840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.925419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.335754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6505.458292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.600748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.724360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.806933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.132863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.220621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.389564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6513.435397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.556766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.631371] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6515.642411] Lustre: Skipped 480 previous similar messages [ 6515.678644] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6515.693967] Lustre: Skipped 480 previous similar messages [ 6515.708418] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6515.721460] Lustre: Skipped 480 previous similar messages [ 6515.732657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.754894] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6515.765811] Lustre: Skipped 480 previous similar messages [ 6516.818538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.911406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.927414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6519.937236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.005650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.994233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.171218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.507909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.646090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6526.571482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.537182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.560704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.497570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6530.508750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.551445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7021.750849] LustreError: 173949:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7021.766056] LustreError: 172146:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7839.734829] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 7839.737161] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7839.745236] LustreError: Skipped 1 previous similar message [ 7839.778720] Lustre: Skipped 15 previous similar messages [ 7839.784445] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7840.742792] LustreError: 226196:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7840.941141] Lustre: server umount lustre-OST0000 complete [ 7845.862066] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7845.876903] Lustre: Skipped 1 previous similar message [ 7850.980325] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7852.045592] Lustre: server umount lustre-MDT0000 complete [ 7856.106043] LustreError: 172126:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7856.129634] LustreError: 172126:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7856.493192] LustreError: 172107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784192744 with bad export cookie 14511317225190001915 [ 7856.510139] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7856.517882] LustreError: 172107:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7856.873953] Lustre: server umount lustre-MDT0001 complete [ 7867.877484] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 7871.186713] Key type lgssc unregistered [ 7871.486747] LNet: 227444:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7871.498473] LNetError: 227444:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7871.521949] LNet: Removed LNI 192.168.204.160@tcp [ 7872.521323] Key type .llcrypt unregistered [ 7872.524502] Key type ._llcrypt unregistered [ 7899.394513] Key type ._llcrypt registered [ 7899.396917] Key type .llcrypt registered [ 7899.574366] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 7901.089612] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7901.235892] alg: No test for adler32 (adler32-zlib) [ 7902.501621] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 7902.872647] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 7904.720178] Key type lgssc registered [ 7906.770172] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7915.886598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7915.945950] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7917.692244] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7923.303808] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 7928.295429] LustreError: 228860:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7928.342338] LustreError: 228860:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7933.413979] LustreError: 228861:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7933.618332] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7933.975932] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7939.049674] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 7947.769982] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7955.303076] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7963.506975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7963.630214] LustreError: 230683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7963.881377] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7969.287617] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7972.763757] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 7982.091871] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7986.968833] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7988.952254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7989.004151] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7989.018669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7989.034288] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7989.035490] LustreError: 228880:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7993.795326] Lustre: lustre-OST0000: Not available for connect from 192.168.204.60@tcp (stopping) [ 7993.828751] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7993.840168] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7994.854521] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7994.863062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7994.876736] Lustre: Skipped 1 previous similar message [ 7997.601740] Lustre: server umount lustre-OST0000 complete [ 8009.096162] LustreError: 228880:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8031.947425] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8032.319990] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8034.422692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8034.447587] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8038.789358] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8044.726078] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8044.763122] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 8046.434409] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8047.982080] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8054.397174] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b8d4556b800.ost_server_uuid 50 [ 8055.913532] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b8d4556b800.ost_server_uuid in FULL state after 0 sec [ 8060.392203] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8060.398741] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8060.434435] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8060.443513] Lustre: Skipped 1 previous similar message [ 8065.512887] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8065.525810] Lustre: Skipped 1 previous similar message [ 8065.922153] Lustre: server umount lustre-OST0000 complete [ 8070.633100] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8070.641194] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8070.650916] Lustre: Skipped 2 previous similar messages [ 8070.666129] Lustre: Skipped 2 previous similar messages [ 8076.545772] Lustre: server umount lustre-MDT0000 complete [ 8080.312538] LustreError: 228840:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784192968 with bad export cookie 15573991195701024969 [ 8080.329136] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8080.790358] Lustre: server umount lustre-MDT0001 complete [ 8092.450320] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 8095.382752] Key type lgssc unregistered [ 8095.697547] LNet: 234071:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8095.725378] LNetError: 234071:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8095.752646] LNet: Removed LNI 192.168.204.160@tcp [ 8096.819183] Key type .llcrypt unregistered [ 8096.821274] Key type ._llcrypt unregistered [ 8128.510462] Key type ._llcrypt registered [ 8128.513497] Key type .llcrypt registered [ 8128.599923] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 8129.398038] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8129.548418] alg: No test for adler32 (adler32-zlib) [ 8130.554596] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 8130.762118] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 8132.432251] Key type lgssc registered [ 8133.511264] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8142.571713] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8142.600178] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8144.048684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8148.883253] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8153.573169] LustreError: 235803:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8153.592621] LustreError: 235803:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8157.669233] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8158.106340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8162.539737] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8165.460589] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8172.651568] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8173.002817] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8180.028476] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8188.423640] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8189.564807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8189.752945] Lustre: lustre-OST0001: new disk, initializing [ 8189.756722] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8189.762510] Lustre: 238110:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8189.866520] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8196.226454] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8197.273756] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8197.290097] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8197.494331] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8207.552893] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8213.344826] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8228.324119] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8228.328453] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8228.333833] Lustre: Skipped 3 previous similar messages [ 8228.343789] Lustre: Skipped 3 previous similar messages [ 8233.098056] Lustre: server umount lustre-MDT0000 complete [ 8233.443896] LustreError: 235804:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8233.484933] LustreError: 235804:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8236.878968] LustreError: 235785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784193124 with bad export cookie 16017171151347609109 [ 8236.886109] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8236.901300] LustreError: 235785:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8237.395609] Lustre: server umount lustre-MDT0001 complete [ 8250.650611] Lustre: server umount lustre-OST0000 complete [ 8263.474389] Lustre: server umount lustre-OST0001 complete [ 8304.215939] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 8316.143774] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8316.602868] LustreError: 241239:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8316.712392] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8321.361131] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8322.026048] LustreError: 241240:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8327.138867] LustreError: 241239:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8330.914758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8331.312861] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8336.862480] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8339.975460] Lustre: 242381:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8347.511279] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8347.853641] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8353.950909] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8356.027544] LustreError: 242736:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8356.062964] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 8362.501805] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 8363.919318] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8371.092855] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8378.361896] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8381.681364] Lustre: 244250:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8413.155415] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8413.159451] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8413.167304] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8415.746831] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8415.768610] Lustre: Skipped 2 previous similar messages [ 8415.769308] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8415.810172] Lustre: Skipped 3 previous similar messages [ 8419.555476] Lustre: server umount mylustre-MDT0000 complete [ 8420.843464] LustreError: 242684:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8420.892878] LustreError: 242684:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 8424.789712] LustreError: 241216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784193312 with bad export cookie 16017171151347611972 [ 8424.790523] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8424.798113] LustreError: 241216:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8425.210176] Lustre: server umount mylustre-MDT0001 complete [ 8438.580193] Lustre: server umount mylustre-OST0000 complete [ 8441.314853] Lustre: 235067:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1784193313/real 1784193313] req@ffff97c5e7d78000 x1870861978210816/t0(0) o400->mylustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1784193329 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 8441.339396] Lustre: mylustre-MDT0001-lwp-OST0001: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8443.051094] Lustre: server umount mylustre-OST0001 complete [ 8468.218786] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 8479.952094] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8480.397063] LustreError: 246580:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8480.494226] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8480.500579] Lustre: Skipped 1 previous similar message [ 8485.564610] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8495.141543] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8499.941346] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8503.345569] Lustre: 247724:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8510.801234] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8518.109441] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8520.436842] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 8525.570232] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 8527.882342] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8535.395151] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8545.820611] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8550.280772] Lustre: 249596:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8559.084482] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8559.088997] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8559.096366] Lustre: Skipped 1 previous similar message [ 8559.120287] Lustre: Skipped 3 previous similar messages [ 8564.205560] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8564.217383] Lustre: Skipped 3 previous similar messages [ 8565.428309] Lustre: server umount tfs-MDT0000 complete [ 8569.328577] LustreError: 246576:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8569.353754] LustreError: 246576:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [ 8569.402906] LustreError: 247323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784193457 with bad export cookie 16017171151347613925 [ 8569.406124] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8569.420595] LustreError: 247323:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 8569.759261] Lustre: server umount tfs-MDT0001 complete [ 8583.783761] Lustre: server umount tfs-OST0000 complete [ 8597.973949] Lustre: server umount tfs-OST0001 complete [ 8625.699144] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 8639.985137] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8640.808879] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8640.819299] Lustre: Skipped 3 previous similar messages [ 8646.428166] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8658.462858] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8664.078136] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8667.003454] Lustre: 252992:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8674.246087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8682.518463] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8684.022526] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 8686.078808] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 8691.636054] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8698.373844] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8707.008398] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8711.351861] Lustre: 254863:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8719.597623] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 05:20:05 (1784193605) [ 8738.285105] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8738.297716] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8738.305656] Lustre: Skipped 3 previous similar messages [ 8738.331272] Lustre: Skipped 2 previous similar messages [ 8742.314798] Lustre: server umount lustre-MDT0000 complete [ 8743.395888] LustreError: 251849:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8743.426984] LustreError: 251849:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 8746.846065] LustreError: 251824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784193634 with bad export cookie 16017171151347615619 [ 8746.858506] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8746.859170] LustreError: 251824:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8756.645862] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8756.657280] Lustre: Skipped 4 previous similar messages [ 8761.313254] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8761.867759] Lustre: server umount lustre-MDT0001 complete [ 8772.693117] Lustre: server umount lustre-OST0000 complete [ 8783.476451] Lustre: server umount lustre-OST0001 complete [ 8792.001876] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 8804.360551] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 8847.745734] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8847.745734] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8847.745734] [ 8847.790281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8848.003741] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8848.031412] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8848.097313] Lustre: lustre-MDT0000: new disk, initializing [ 8848.149220] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8848.157873] Lustre: Skipped 3 previous similar messages [ 8848.169357] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8853.250914] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8864.808386] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8864.808386] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8864.808386] [ 8864.846919] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8865.007890] Lustre: 258050:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8865.095542] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8865.099690] Lustre: Skipped 1 previous similar message [ 8865.185397] Lustre: lustre-MDT0001: new disk, initializing [ 8865.319432] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8865.333434] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8869.759918] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8884.408771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8884.803669] Lustre: lustre-OST0000: new disk, initializing [ 8884.815064] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8884.822325] Lustre: 259633:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8886.220819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8886.248616] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8886.452064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8893.966264] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8906.694033] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8906.962131] Lustre: lustre-OST0001: new disk, initializing [ 8906.965668] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8906.980910] Lustre: 260507:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8908.342170] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8908.358661] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8908.532081] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8914.406720] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 8929.097911] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 05:23:34 (1784193814) [ 8950.805510] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 05:23:55 (1784193835) [ 8968.675185] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8968.682937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8968.701320] Lustre: Skipped 4 previous similar messages [ 8968.718028] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8974.931824] Lustre: server umount lustre-OST0000 complete [ 8985.503982] Lustre: server umount lustre-MDT0000 complete [ 8990.132771] LustreError: 258040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784193877 with bad export cookie 16017171151347617173 [ 8990.151407] LustreError: 258040:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8990.153127] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8996.766704] Lustre: server umount lustre-MDT0001 complete [ 9009.293736] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 9022.522736] Lustre: server umount lustre-OST0001 complete [ 9031.928742] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 9050.984345] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 9089.488520] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9089.673274] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9089.705726] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9089.757393] Lustre: lustre-MDT0000: new disk, initializing [ 9089.819580] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9094.745754] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9105.564414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9105.729915] Lustre: 265064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9105.800367] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9105.804956] Lustre: Skipped 1 previous similar message [ 9105.875387] Lustre: lustre-MDT0001: new disk, initializing [ 9105.974599] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9105.977661] Lustre: Skipped 4 previous similar messages [ 9106.005911] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9106.019476] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9110.489839] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9120.604852] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9126.944564] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9135.341343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9135.586060] Lustre: lustre-OST0000: new disk, initializing [ 9135.591410] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9135.601906] Lustre: 267033:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9137.632290] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9137.656111] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9137.838451] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9143.208386] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9154.401574] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9163.233637] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9163.244803] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9163.267438] Lustre: Skipped 5 previous similar messages [ 9163.280661] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9163.289414] Lustre: Skipped 7 previous similar messages [ 9168.177730] Lustre: server umount lustre-OST0000 complete [ 9172.449709] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9172.465979] LustreError: Skipped 1 previous similar message [ 9178.490505] Lustre: server umount lustre-MDT0000 complete [ 9181.668854] LustreError: 267704:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9181.686635] LustreError: 267704:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 16 previous similar messages [ 9182.848708] LustreError: 265054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194070 with bad export cookie 16017171151347618636 [ 9182.852801] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9182.859230] LustreError: 265054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9183.253309] Lustre: server umount lustre-MDT0001 complete [ 9195.734915] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 9198.517241] Key type lgssc unregistered [ 9198.772557] LNet: 268794:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9198.782434] LNetError: 268794:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9198.812671] LNet: Removed LNI 192.168.204.160@tcp [ 9199.694417] Key type .llcrypt unregistered [ 9199.699506] Key type ._llcrypt unregistered [ 9228.109352] Key type ._llcrypt registered [ 9228.112073] Key type .llcrypt registered [ 9228.249960] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 9229.827649] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9229.854059] alg: No test for adler32 (adler32-zlib) [ 9231.083545] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 9231.389381] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 9233.057378] Key type lgssc registered [ 9234.461918] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9243.084160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9243.144743] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9244.685280] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9249.153155] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9254.883429] LustreError: 270257:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9254.908732] LustreError: 270257:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9258.066221] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9258.539961] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9264.055505] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9272.363827] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9279.299476] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9286.623404] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9287.054537] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9290.188335] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9295.362548] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9305.141294] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9306.230048] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9308.652995] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9308.659851] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9308.669392] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9308.683925] Lustre: Skipped 1 previous similar message [ 9310.725572] Lustre: server umount lustre-OST0000 complete [ 9318.886236] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9318.899210] Lustre: Skipped 1 previous similar message [ 9318.906880] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9318.913403] Lustre: Skipped 1 previous similar message [ 9321.446724] Lustre: server umount lustre-MDT0000 complete [ 9324.004061] LustreError: 270254:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9324.030741] LustreError: 270254:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9325.649835] LustreError: 270237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194213 with bad export cookie 8204381448909979891 [ 9325.654373] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9325.661376] LustreError: 270237:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9326.133919] Lustre: server umount lustre-MDT0001 complete [ 9339.319321] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 9342.140330] Key type lgssc unregistered [ 9342.542538] LNet: 273843:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9342.555454] LNetError: 273843:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9342.566891] LNet: Removed LNI 192.168.204.160@tcp [ 9343.479702] Key type .llcrypt unregistered [ 9343.491750] Key type ._llcrypt unregistered [ 9360.779724] Key type ._llcrypt registered [ 9360.782528] Key type .llcrypt registered [ 9360.901177] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 9362.158429] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9362.247655] alg: No test for adler32 (adler32-zlib) [ 9363.421376] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 9363.716522] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 9365.456903] Key type lgssc registered [ 9366.688312] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9376.586989] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9376.607430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9392.162313] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9397.280725] LustreError: 275038:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.160@tcp: failed processing log, type 4: rc = -110 [ 9423.968205] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9431.392212] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9440.274454] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9442.629939] Lustre: server umount lustre-OST0000 complete [ 9461.440358] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [ 9464.261106] Key type lgssc unregistered [ 9464.636165] LNet: 276668:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9464.649391] LNetError: 276668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9464.690312] LNet: Removed LNI 192.168.204.160@tcp [ 9465.690284] Key type .llcrypt unregistered [ 9465.700509] Key type ._llcrypt unregistered [ 9504.336914] Key type ._llcrypt registered [ 9504.339643] Key type .llcrypt registered [ 9504.441387] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 9505.535422] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9505.600357] alg: No test for adler32 (adler32-zlib) [ 9506.682961] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [ 9506.995279] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [ 9508.656209] Key type lgssc registered [ 9510.216962] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9523.421432] loop0: detected capacity change from 0 to 204800000 [ 9523.813161] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9523.825662] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9524.242292] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9525.720789] loop0: detected capacity change from 0 to 204800000 [ 9525.951304] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9525.971368] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9526.410340] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9527.693377] loop0: detected capacity change from 0 to 204800000 [ 9527.936045] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9527.951348] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9528.085396] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9529.415406] loop0: detected capacity change from 0 to 204800000 [ 9529.654652] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9529.671119] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9529.861619] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9531.105067] loop0: detected capacity change from 0 to 204800000 [ 9531.135870] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9536.015418] loop0: detected capacity change from 0 to 204800000 [ 9536.047096] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9539.954443] loop0: detected capacity change from 0 to 204800000 [ 9539.981440] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9544.035390] loop0: detected capacity change from 0 to 204800000 [ 9544.064568] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9548.407223] loop0: detected capacity change from 0 to 204800000 [ 9548.544201] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9548.619668] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9548.687423] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9548.698274] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9548.713372] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9548.727955] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9548.749464] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9548.774438] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9548.791342] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9548.801386] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9548.812199] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9548.840351] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9548.857103] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9548.869205] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9548.884982] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9548.900966] Lustre: 280082:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9555.465972] Lustre: server umount MGS complete [ 9556.441634] loop0: detected capacity change from 0 to 204800000 [ 9556.576993] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9557.000250] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9557.045558] Lustre: lustre-MDT0000: Aborting client recovery [ 9558.161927] loop1: detected capacity change from 0 to 204800000 [ 9558.408903] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9558.505340] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9558.524603] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9558.544729] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9558.553423] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9558.571404] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9558.592934] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9558.620651] Lustre: 280515:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9558.662452] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9558.819404] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9558.890291] Lustre: lustre-MDT0001: Aborting client recovery [ 9559.689879] loop2: detected capacity change from 0 to 204800000 [ 9559.850109] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9559.944660] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9559.952331] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9559.960776] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9559.982657] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9559.997993] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9560.008567] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9560.016915] Lustre: 280597:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9560.186638] Lustre: Found index 0 for lustre-OST0000, updating log [ 9560.254654] Lustre: 280615:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9560.333690] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9560.361567] Lustre: lustre-OST0000: Aborting recovery [ 9561.224247] loop3: detected capacity change from 0 to 204800000 [ 9561.451374] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9561.541966] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9561.565793] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9561.584715] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9561.608349] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9561.621866] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9561.641174] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9561.652402] Lustre: 280680:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9561.687407] Lustre: Found index 1 for lustre-OST0001, updating log [ 9561.737649] Lustre: 280686:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9561.829446] Lustre: lustre-OST0001: Aborting recovery [ 9566.963927] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9566.966115] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9567.006596] LustreError: 280459:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9567.010959] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9567.020216] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9569.001794] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9569.006075] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9570.613908] LustreError: 280426:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 13390us: rc = -2 [ 9571.118761] LustreError: 280426:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 453us: rc = -2 [ 9571.148114] LustreError: 280426:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 16 previous similar messages [ 9584.609101] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9584.614839] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9584.615492] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9584.634904] Lustre: Skipped 3 previous similar messages [ 9584.675514] Lustre: Skipped 3 previous similar messages [ 9589.741410] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9589.757723] Lustre: Skipped 3 previous similar messages [ 9590.945319] Lustre: server umount lustre-MDT0000 complete [ 9592.192630] LustreError: 280409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194479 with bad export cookie 11281598729451919536 [ 9592.232169] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9594.849133] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9594.862541] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9594.862817] LustreError: 280425:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9594.862824] LustreError: 280425:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9594.873916] Lustre: Skipped 1 previous similar message [ 9594.919885] Lustre: Skipped 1 previous similar message [ 9597.857244] LustreError: 280424:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9597.859978] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9597.893363] LustreError: 280424:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9597.929032] Lustre: Skipped 1 previous similar message [ 9598.761452] Lustre: server umount lustre-MDT0001 complete [ 9605.661468] Lustre: server umount lustre-OST0000 complete [ 9612.906547] Lustre: server umount lustre-OST0001 complete [ 9621.561607] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 05:35:07 (1784194507) [ 9650.819858] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [ 9663.514820] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [ 9704.683432] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9704.879367] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9704.900592] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9704.965609] Lustre: lustre-MDT0000: new disk, initializing [ 9705.031939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9705.037399] Lustre: Skipped 1 previous similar message [ 9705.050813] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9710.105620] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9720.983305] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9721.090616] Lustre: 284752:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9721.168236] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9721.177387] Lustre: Skipped 1 previous similar message [ 9721.303399] Lustre: lustre-MDT0001: new disk, initializing [ 9721.434832] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9721.478819] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9721.500185] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9726.440330] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9736.849769] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9742.633398] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9750.743771] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9751.153648] Lustre: lustre-OST0000: new disk, initializing [ 9751.158641] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9751.169502] Lustre: 286720:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9751.291857] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9758.003933] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9759.358053] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9759.371477] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9759.507809] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9769.328435] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9775.883666] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9788.506514] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9794.935934] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9841.643120] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9841.647545] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9841.654490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9841.660305] Lustre: Skipped 1 previous similar message [ 9846.603535] Lustre: server umount lustre-OST0000 complete [ 9851.873351] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9851.880303] Lustre: Skipped 1 previous similar message [ 9851.886562] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9851.891307] Lustre: Skipped 1 previous similar message [ 9857.544116] Lustre: server umount lustre-MDT0000 complete [ 9861.999946] LustreError: 287968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194749 with bad export cookie 11281598729451922238 [ 9862.002492] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9862.029900] LustreError: 287968:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 9862.400686] Lustre: server umount lustre-MDT0001 complete [ 9871.870472] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9876.681666] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9882.494567] Lustre: server umount MGS complete [ 9895.266635] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9895.806556] LustreError: 290585:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9895.833435] LustreError: 290585:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9895.836991] LustreError: 290598:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9895.994684] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9901.037800] LustreError: 290586:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9902.491440] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9906.149653] LustreError: 290585:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9911.863073] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9916.592107] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9925.390526] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9931.633477] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9938.891524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9939.215659] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9939.219991] Lustre: Skipped 1 previous similar message [ 9946.219888] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [ 9954.151678] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9987.559497] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9987.562828] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9987.571517] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9987.571526] Lustre: Skipped 2 previous similar messages [ 9987.576075] LustreError: Skipped 1 previous similar message [ 9987.598873] Lustre: Skipped 2 previous similar messages [ 9990.677077] Lustre: server umount lustre-OST0000 complete [ 9994.211163] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9994.221242] LustreError: Skipped 1 previous similar message [ 9997.794614] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9997.814032] Lustre: Skipped 1 previous similar message [10000.588379] Lustre: server umount lustre-MDT0000 complete [10002.922689] LustreError: 290581:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10002.942812] LustreError: 290581:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10004.536457] LustreError: 290567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784194892 with bad export cookie 11281598729451924072 [10004.555081] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10004.556008] LustreError: 290567:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [10004.998790] Lustre: server umount lustre-MDT0001 complete [10018.139508] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [10020.969175] Key type lgssc unregistered [10021.364484] LNet: 294535:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10021.370960] LNetError: 294535:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10021.387365] LNet: Removed LNI 192.168.204.160@tcp [10022.345683] Key type .llcrypt unregistered [10022.349967] Key type ._llcrypt unregistered [10064.001600] Key type ._llcrypt registered [10064.007153] Key type .llcrypt registered [10064.185888] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [10080.427075] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [10081.313750] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10081.414708] alg: No test for adler32 (adler32-zlib) [10082.557760] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [10082.837322] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [10084.529239] Key type lgssc registered [10086.029039] Lustre: Echo OBD driver; http://www.lustre.org/ [10121.221350] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10121.238119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10122.510175] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10122.560867] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10122.668780] Lustre: lustre-MDT0000: new disk, initializing [10122.766433] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10122.804513] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10127.017418] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10136.918489] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10137.052619] Lustre: 298499:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10137.130313] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10137.134301] Lustre: Skipped 1 previous similar message [10137.189357] Lustre: lustre-MDT0001: new disk, initializing [10137.275199] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10137.344987] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10137.366207] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10141.420975] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10150.003596] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10155.114987] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10162.625143] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10162.976108] Lustre: lustre-OST0000: new disk, initializing [10162.982414] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10162.995033] Lustre: 300465:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10163.111632] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10169.842537] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10172.496690] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10172.511961] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10172.653302] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10180.417311] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10187.761571] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10202.434852] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10209.914653] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10220.176688] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10259.937424] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10259.942409] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10259.945410] LustreError: Skipped 1 previous similar message [10259.946638] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10259.982700] Lustre: Skipped 1 previous similar message [10263.963611] Lustre: server umount lustre-OST0000 complete [10270.180365] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10270.181762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10270.191752] Lustre: Skipped 1 previous similar message [10270.193912] Lustre: Skipped 1 previous similar message [10274.140197] Lustre: server umount lustre-MDT0000 complete [10278.079475] LustreError: 299292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195165 with bad export cookie 8725538472962827058 [10278.080670] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10278.086997] LustreError: 299292:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10278.404122] Lustre: server umount lustre-MDT0001 complete [10287.380051] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10291.756794] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10296.815825] Lustre: server umount MGS complete [10308.101297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10308.524441] LustreError: 304336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10308.540272] LustreError: 304336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10308.551413] LustreError: 304349:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10308.610603] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10313.235586] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10313.697346] LustreError: 304337:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10318.831038] LustreError: 304336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10323.580975] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10324.041027] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10330.302859] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10339.431686] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10345.302142] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10354.099493] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10354.593307] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10361.357472] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10370.288940] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10415.080810] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10415.086213] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10415.104794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10415.109416] LustreError: Skipped 1 previous similar message [10415.135501] Lustre: Skipped 1 previous similar message [10416.453821] Lustre: server umount lustre-OST0000 complete [10421.218426] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10421.229675] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10421.245586] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10421.251190] Lustre: Skipped 1 previous similar message [10425.319713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10425.330940] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10425.358858] Lustre: Skipped 1 previous similar message [10426.857625] Lustre: server umount lustre-MDT0000 complete [10430.437532] LustreError: 304336:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10430.481901] LustreError: 304336:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [10431.593393] LustreError: 307088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784195319 with bad export cookie 8725538472962828997 [10431.601380] LustreError: 307088:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10431.601719] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10432.189491] Lustre: server umount lustre-MDT0001 complete [10443.817456] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [10447.155422] Key type lgssc unregistered [10447.546838] LNet: 308309:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10447.562043] LNetError: 308309:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10447.585036] LNet: Removed LNI 192.168.204.160@tcp [10448.529074] Key type .llcrypt unregistered [10448.534262] Key type ._llcrypt unregistered [10470.975741] Key type ._llcrypt registered [10470.977826] Key type .llcrypt registered [10471.117505] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [10486.136932] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [10487.146520] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10487.313061] alg: No test for adler32 (adler32-zlib) [10488.430584] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [10488.728901] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [10490.456512] Key type lgssc registered [10491.530828] Lustre: Echo OBD driver; http://www.lustre.org/ [10516.781463] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [10533.935642] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10533.951673] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10535.220202] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10535.281261] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10535.427828] Lustre: lustre-MDT0000: new disk, initializing [10535.524347] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10535.539441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10540.412295] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [10552.655317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10553.157286] Lustre: lustre-OST0000: new disk, initializing [10553.161480] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10553.164279] Lustre: Skipped 1 previous similar message [10553.171899] Lustre: 312380:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10553.256803] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10559.162358] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10559.179531] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10559.241466] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10561.674281] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11568.098478] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11568.112671] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11568.617903] Lustre: server umount lustre-MDT0000 complete [11572.726839] LustreError: 311566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196460 with bad export cookie 12310489118404971894 [11572.731538] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11572.739445] LustreError: 311566:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [11572.963585] Lustre: server umount lustre-OST0000 complete [11599.410541] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [11608.910762] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [11650.636302] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11650.890254] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11650.922410] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11651.032418] Lustre: lustre-MDT0000: new disk, initializing [11651.179650] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11651.256659] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11656.587384] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11667.963788] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11668.057467] Lustre: 316752:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11668.127254] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11668.131469] Lustre: Skipped 1 previous similar message [11668.205363] Lustre: lustre-MDT0001: new disk, initializing [11668.331683] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11668.374969] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11668.397649] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11673.514968] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11685.710553] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11691.981614] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11700.011563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11700.252388] Lustre: lustre-OST0000: new disk, initializing [11700.255994] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11700.262068] Lustre: 318722:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11700.406875] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11707.410055] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11709.546717] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11709.573344] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11709.770802] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11718.909572] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11723.790627] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11724.046971] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11728.955664] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11729.148779] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11735.531039] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11735.533847] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11735.543027] LustreError: Skipped 1 previous similar message [11735.544682] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11735.573246] Lustre: Skipped 1 previous similar message [11737.436378] Lustre: server umount lustre-OST0000 complete [11745.250637] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11745.260279] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11745.284404] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11745.289948] Lustre: Skipped 1 previous similar message [11749.017568] Lustre: server umount lustre-MDT0000 complete [11753.594825] LustreError: 316745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196641 with bad export cookie 12310489118407912713 [11753.599977] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11753.607791] LustreError: 316745:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11753.977613] Lustre: server umount lustre-MDT0001 complete [11758.331448] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [11760.489694] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 06:10:45 (1784196645) [11769.004195] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11769.645307] LustreError: 321036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11769.687900] LustreError: 321036:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [11769.824444] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11774.947732] LustreError: 321037:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11775.161315] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11780.070095] LustreError: 321036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11785.194615] LustreError: 321037:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11785.459284] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11786.016734] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11790.938847] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11800.266401] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11806.801852] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11814.216708] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11814.705900] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11822.447834] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11831.494778] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11840.163410] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11840.401767] Lustre: lustre-OST0001: new disk, initializing [11840.407538] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [11840.414556] Lustre: 323873:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11840.503782] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11847.531489] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11849.852899] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [11849.868524] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [11849.917981] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11849.931859] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11850.984577] LustreError: 324333:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11851.004677] LustreError: 324333:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [11852.008442] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11854.117699] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11854.149957] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [11858.277949] LustreError: 324333:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [11858.298682] LustreError: 324333:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [11859.336449] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11859.588424] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11866.154238] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff8b8d58a8f000.ost_server_uuid 50 [11866.475269] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [11866.482127] LustreError: 324337:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [11867.645318] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff8b8d58a8f000.ost_server_uuid in FULL state after 0 sec [11873.625731] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff8b8d58a8f000.ost_server_uuid 50 [11875.462459] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff8b8d58a8f000.ost_server_uuid in FULL state after 0 sec [11877.908067] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [11893.217758] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11893.238125] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11893.255985] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [11893.259824] Lustre: Skipped 2 previous similar messages [11893.292808] Lustre: Skipped 2 previous similar messages [11897.460766] Lustre: server umount lustre-OST0001 complete [11899.318166] LustreError: 322858:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 192.168.204.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11906.533365] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11906.560694] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11906.563880] Lustre: Skipped 1 previous similar message [11906.582055] Lustre: Skipped 4 previous similar messages [11910.323246] Lustre: server umount lustre-OST0000 complete [11916.774691] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11916.791783] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11916.797931] Lustre: Skipped 1 previous similar message [11920.588647] Lustre: server umount lustre-MDT0000 complete [11921.890888] LustreError: 325410:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11921.905983] LustreError: 325410:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [11924.380909] LustreError: 323872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784196812 with bad export cookie 12310489118407913602 [11924.396766] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11924.400576] LustreError: 323872:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11924.846179] Lustre: server umount lustre-MDT0001 complete [11935.950461] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [11938.569270] Key type lgssc unregistered [11938.848593] LNet: 326565:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11938.862042] LNetError: 326565:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11938.897547] LNet: Removed LNI 192.168.204.160@tcp [11939.864399] Key type .llcrypt unregistered [11939.866549] Key type ._llcrypt unregistered [11964.996823] Key type ._llcrypt registered [11964.999994] Key type .llcrypt registered [11965.091897] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [11966.085668] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11966.119647] alg: No test for adler32 (adler32-zlib) [11967.312964] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [11967.653492] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [11969.344148] Key type lgssc registered [11970.702308] Lustre: Echo OBD driver; http://www.lustre.org/ [11979.080888] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11979.099062] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11980.705974] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11985.615453] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [11985.894536] LustreError: 327983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11985.910411] LustreError: 327983:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [11991.011444] LustreError: 327984:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11994.091948] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11996.131835] LustreError: 327983:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12001.255843] LustreError: 327984:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12001.402639] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12001.741098] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12006.573625] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12015.797923] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12016.087820] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12022.166827] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12025.350505] LustreError: 329665:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12025.369807] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12029.251850] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12035.895370] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12036.209508] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12041.239135] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12043.590114] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12051.273255] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12058.017988] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12058.282185] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12164.069957] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12164.070131] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12164.082284] LustreError: Skipped 1 previous similar message [12164.089653] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12164.089743] Lustre: Skipped 1 previous similar message [12169.199234] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12169.206476] Lustre: Skipped 1 previous similar message [12174.305339] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12174.311991] Lustre: Skipped 1 previous similar message [12177.378584] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12177.533385] Lustre: server umount lustre-OST0001 complete [12179.429970] LustreError: 330178:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12179.444921] LustreError: 330178:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [12184.037655] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12184.047702] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12184.065345] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12189.667251] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12189.681311] Lustre: Skipped 2 previous similar messages [12196.832201] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12197.034783] Lustre: server umount lustre-OST0000 complete [12201.442814] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12201.451455] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12201.465155] Lustre: Skipped 1 previous similar message [12201.478959] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12201.483587] Lustre: Skipped 2 previous similar messages [12205.030242] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12206.548849] Lustre: server umount lustre-MDT0000 complete [12209.807920] LustreError: 331672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197097 with bad export cookie 2413555652916070679 [12209.810594] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12209.818344] LustreError: 331672:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12210.166980] Lustre: server umount lustre-MDT0001 complete [12220.800281] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [12223.924671] Key type lgssc unregistered [12224.225729] LNet: 332932:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12224.234726] LNetError: 332932:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12224.254611] LNet: Removed LNI 192.168.204.160@tcp [12225.131251] Key type .llcrypt unregistered [12225.132990] Key type ._llcrypt unregistered [12267.982640] Key type ._llcrypt registered [12267.985104] Key type .llcrypt registered [12268.090516] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [12268.919652] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12268.930115] alg: No test for adler32 (adler32-zlib) [12269.904402] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [12270.152620] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [12271.808167] Key type lgssc registered [12272.893396] Lustre: Echo OBD driver; http://www.lustre.org/ [12280.470745] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12280.487020] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12281.840074] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12285.509900] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12292.066654] LustreError: 335222:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12292.089607] LustreError: 335222:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12292.344055] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12292.643303] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12295.822416] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12298.336287] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12303.595272] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12303.872599] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12306.924231] LustreError: 336712:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12306.933955] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12308.601108] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12312.043886] LustreError: 336710:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12316.436407] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12316.682809] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12317.682523] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12321.761428] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12328.836159] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12336.001979] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12348.403588] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12348.407357] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12348.418384] Lustre: Skipped 2 previous similar messages [12348.422447] Lustre: Skipped 3 previous similar messages [12351.623548] Lustre: server umount lustre-MDT0000 complete [12353.512825] LustreError: 338047:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12353.524716] LustreError: 338047:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12354.097389] LustreError: 338230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197241 with bad export cookie 7874187622579434526 [12354.107863] LustreError: 338230:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12354.108980] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12354.311366] Lustre: server umount lustre-MDT0001 complete [12366.889226] Lustre: server umount lustre-OST0000 complete [12380.458489] Lustre: server umount lustre-OST0001 complete [12394.142700] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [12403.351143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12403.657770] LustreError: 340489:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12403.738855] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12407.730349] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12413.925407] LustreError: 340490:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12413.945220] LustreError: 340490:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12415.901844] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12416.233172] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12420.236513] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12422.797631] Lustre: 341629:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12428.474835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12433.330484] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12433.902894] LustreError: 341982:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12433.916791] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12433.919843] LustreError: 341982:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12440.175078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12440.388912] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12440.393101] Lustre: Skipped 1 previous similar message [12441.404386] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12445.560932] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12452.278995] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12455.450403] Lustre: 343501:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12462.053954] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12462.061317] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12462.073410] Lustre: Skipped 3 previous similar messages [12462.087530] Lustre: Skipped 4 previous similar messages [12465.379077] Lustre: server umount lustre-MDT0000 complete [12467.170986] LustreError: 341221:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12467.192396] LustreError: 341221:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [12468.973562] LustreError: 343504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197356 with bad export cookie 7874187622579436227 [12468.981543] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12468.989598] LustreError: 343504:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12469.355302] Lustre: server umount lustre-MDT0001 complete [12482.567901] Lustre: server umount lustre-OST0000 complete [12496.191915] Lustre: server umount lustre-OST0001 complete [12499.817810] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12501.074644] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12502.524296] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 06:23:08 (1784197388) [12521.569533] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [12530.658670] loop0: detected capacity change from 0 to 1099511627776 [12531.146102] SGI XFS with ACLs, security attributes, quota, no debug enabled [12531.151923] *** VALIDATE xfs *** [12531.159673] XFS (loop0): Mounting V5 Filesystem [12531.173733] XFS (loop0): Ending clean mount [12532.384966] loop1: detected capacity change from 0 to 18691697672192 [12616.216789] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [12624.973252] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [12659.456652] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12659.726398] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12659.790805] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12659.871996] Lustre: lustre-MDT0000: new disk, initializing [12659.958798] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12659.995871] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12663.984067] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12674.023987] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12674.125418] Lustre: 348632:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12674.190666] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12674.194322] Lustre: Skipped 1 previous similar message [12674.262838] Lustre: lustre-MDT0001: new disk, initializing [12674.364627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12674.384404] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12678.944274] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12687.950942] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12693.585956] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12701.081775] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12701.339501] Lustre: lustre-OST0000: new disk, initializing [12701.343360] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12701.350545] Lustre: 350602:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12702.725272] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12702.734927] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12702.863474] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12706.886723] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12715.859749] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12720.446856] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12720.635678] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12725.155187] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12725.405804] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12728.289871] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12728.301162] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12728.322834] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12728.804171] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12733.592383] Lustre: server umount lustre-OST0000 complete [12739.043901] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12739.058205] Lustre: Skipped 1 previous similar message [12739.062968] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12739.066066] Lustre: Skipped 1 previous similar message [12744.162936] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12751.328108] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12751.557084] Lustre: server umount lustre-MDT0000 complete [12754.406091] LustreError: 348640:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12754.435690] LustreError: 348640:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12755.217658] LustreError: 348624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197642 with bad export cookie 7874187622579437781 [12755.219804] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12755.230372] LustreError: 348624:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12755.542959] Lustre: server umount lustre-MDT0001 complete [12763.231458] XFS (loop0): Unmounting Filesystem [12767.380797] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 06:27:33 (1784197653) [12773.716105] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12774.058856] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12774.066577] Lustre: Skipped 2 previous similar messages [12777.600220] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12785.475086] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12789.939104] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12796.669461] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12801.932800] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12807.962836] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12813.381913] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12820.052656] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12828.644807] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12828.651066] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12828.667440] Lustre: Skipped 1 previous similar message [12828.688610] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12828.695726] Lustre: Skipped 3 previous similar messages [12833.566899] Lustre: server umount lustre-OST0000 complete [12837.346097] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12837.354159] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12837.369186] Lustre: Skipped 1 previous similar message [12843.479533] Lustre: server umount lustre-MDT0000 complete [12847.006152] LustreError: 355007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784197734 with bad export cookie 7874187622579438663 [12847.016717] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12847.023323] LustreError: 355007:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12847.409442] Lustre: server umount lustre-MDT0001 complete [12857.750450] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [12860.426633] Key type lgssc unregistered [12860.738681] LNet: 356792:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12860.756424] LNetError: 356792:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12860.788668] LNet: Removed LNI 192.168.204.160@tcp [12861.675186] Key type .llcrypt unregistered [12861.677604] Key type ._llcrypt unregistered [12886.911086] Key type ._llcrypt registered [12886.916459] Key type .llcrypt registered [12887.036406] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [12888.172300] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12888.207457] alg: No test for adler32 (adler32-zlib) [12889.214322] Lustre: Lustre: Build Version: 2.17.54_162_gdc75e25 [12889.416825] LNet: Added LNI 192.168.204.160@tcp [8/256/0/180] [12891.080796] Key type lgssc registered [12891.980203] Lustre: Echo OBD driver; http://www.lustre.org/ [12898.468781] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12898.497276] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12899.879834] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12903.739642] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12910.052948] LustreError: 358364:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12910.063780] LustreError: 358364:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12911.776546] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12912.096426] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12915.970379] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12922.598246] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12927.414667] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12933.631270] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12934.031366] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12939.597324] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12945.348476] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12953.022294] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.60@tcp (stopping) [12953.058503] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12953.065701] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12954.596384] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12954.597099] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [12954.618331] Lustre: Skipped 2 previous similar messages [12958.122452] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.60@tcp (stopping) [12963.243322] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.60@tcp (stopping) [12963.258101] Lustre: Skipped 2 previous similar messages [12966.368141] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12966.509770] Lustre: server umount lustre-MDT0001 complete [12968.382193] LustreError: 358361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12969.960638] LustreError: 358364:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12969.974814] LustreError: 358364:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12973.487653] LustreError: 358361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12975.534963] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12975.634296] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12975.640274] Lustre: Found index 1 for lustre-MDT0001, updating log [12975.681471] Lustre: 358354:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12975.978089] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12979.790406] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [12981.233994] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12984.818619] LustreError: 361968:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12984.825116] LustreError: 361968:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12984.833513] LustreError: 358357:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12984.875391] LustreError: 361975:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.160@tcp: cfg command failed: rc = -17 [12984.890268] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [12984.890268] [13285.850559] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.60@tcp (stopping) [13285.857581] Lustre: Skipped 2 previous similar messages [13288.420874] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13288.424510] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13288.445531] Lustre: Skipped 1 previous similar message [13290.692937] Lustre: server umount lustre-MDT0001 complete [13290.925271] LustreError: 358709:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.204.60@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [13290.951879] LustreError: 358709:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13299.622149] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13299.712544] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13299.717763] Lustre: Found index 1 for lustre-MDT0001, updating log [13299.763751] Lustre: 358353:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13300.019328] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13303.893488] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [13305.328032] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13309.930981] LustreError: 371501:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13309.939118] LustreError: 371501:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13309.943779] LustreError: 358357:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13309.949814] LustreError: 358357:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13309.982511] LustreError: 371504:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13309.986781] LustreError: 371504:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13309.990295] LustreError: 371504:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.160@tcp: cfg command failed: rc = -17 [13309.998644] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.160@tcp [13309.998644] [13608.380408] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.60@tcp (stopping) [13608.391045] Lustre: Skipped 2 previous similar messages [13612.518424] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13612.526929] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13613.821801] Lustre: server umount lustre-MDT0001 complete [13617.639633] LustreError: 359094:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13617.659607] LustreError: 359094:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [13622.834348] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13622.910467] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13622.914675] Lustre: Found index 1 for lustre-MDT0001, updating log [13622.950497] Lustre: 358353:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13623.224157] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13626.744986] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [13628.406099] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13629.941163] LustreError: 380957:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13629.947625] LustreError: 380957:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13629.963039] LustreError: 358357:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13629.967639] LustreError: 358357:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13630.016622] LustreError: 380960:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [13630.032367] LustreError: 380960:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.160@tcp: cfg command failed: rc = -114 [13630.041074] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [13630.041074] [13945.829861] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13945.830566] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [13945.838452] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13945.838464] Lustre: Skipped 3 previous similar messages [13945.842473] Lustre: Skipped 2 previous similar messages [13950.222903] Lustre: server umount lustre-MDT0000 complete [13950.949891] LustreError: 358361:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13950.967076] LustreError: 358361:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [13952.946512] LustreError: 358344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198840 with bad export cookie 7440338948250457157 [13952.948977] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13952.953833] LustreError: 358344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [13956.067309] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13956.074648] Lustre: Skipped 1 previous similar message [13967.329213] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13967.510977] Lustre: server umount lustre-MDT0001 complete [13969.947056] Lustre: server umount lustre-OST0000 complete [13975.844626] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [13982.383747] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [14004.370340] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:48:10 (1784198890) [14008.242656] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14008.384484] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14008.407092] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14008.465734] Lustre: lustre-MDT0000: new disk, initializing [14008.512070] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14008.529801] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14010.970405] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [14017.505766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14017.586322] Lustre: 392686:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14017.640168] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14017.643416] Lustre: Skipped 1 previous similar message [14017.695075] Lustre: lustre-MDT0001: new disk, initializing [14017.762118] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14017.792652] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14017.799846] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14020.613533] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [14026.769229] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14030.080414] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14034.371203] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14034.539905] Lustre: lustre-OST0000: new disk, initializing [14034.543471] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14034.550834] Lustre: 394654:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14034.613326] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14038.187678] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_default_debug -1 all [14043.668124] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14043.673618] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14043.732104] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14043.818947] Lustre: DEBUG MARKER: oleg460-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14048.740614] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14048.745314] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14048.746458] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14048.755105] Lustre: Skipped 2 previous similar messages [14048.762209] Lustre: Skipped 7 previous similar messages [14054.260401] Lustre: server umount lustre-MDT0000 complete [14056.289307] LustreError: 392677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1784198944 with bad export cookie 7440338948250459110 [14056.292100] LustreError: MGC192.168.204.160@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14056.293355] LustreError: 393491:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14056.295286] LustreError: 392677:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [14056.496375] Lustre: server umount lustre-MDT0001 complete [14069.204398] Lustre: server umount lustre-OST0000 complete [14087.593537] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing set_hostid [14094.212680] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing load_modules_local [14113.636633] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 13808 sec ========= 06:49:59 (1784198999) [14114.690549] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:50:00 (1784199000) === [14116.973057] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:50:03 (1784199003) === [14141.971370] Lustre: DEBUG MARKER: oleg460-server.virtnet: executing unload_modules_local [14143.788931] Key type lgssc unregistered [14143.975760] LNet: 401013:0:(lib-ptl.c:967:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14143.982572] LNetError: 401013:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14143.991728] LNet: Removed LNI 192.168.204.160@tcp [14144.531499] Key type .llcrypt unregistered [14144.534347] Key type ._llcrypt unregistered