[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 512471632 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2399.998 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2895240K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.002341] x2apic enabled [ 0.003007] Switched APIC routing to physical x2apic. [ 0.004015] kvm-guest: setup PV IPIs [ 0.007450] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 0.008019] Calibrating delay loop (skipped) preset value.. 4799.99 BogoMIPS (lpj=2399998) [ 0.009013] pid_max: default: 32768 minimum: 301 [ 0.010125] LSM: Security Framework initializing [ 0.011052] Yama: becoming mindful. [ 0.012035] SELinux: Initializing. [ 0.013059] *** VALIDATE selinux *** [ 0.022210] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.026646] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028150] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.029104] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030096] *** VALIDATE tmpfs *** [ 0.032149] *** VALIDATE proc *** [ 0.033232] *** VALIDATE cgroup *** [ 0.034009] *** VALIDATE cgroup2 *** [ 0.036046] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.038023] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.039008] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.040027] Spectre V2 : User space: Vulnerable [ 0.041005] Speculative Store Bypass: Vulnerable [ 0.044133] debug: unmapping init [mem 0xffffffff96a59000-0xffffffff96a60fff] [ 0.046154] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047693] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048024] ... version: 2 [ 0.049012] ... bit width: 48 [ 0.050015] ... generic registers: 4 [ 0.051012] ... value mask: 0000ffffffffffff [ 0.052015] ... max period: 00007fffffffffff [ 0.053015] ... fixed-purpose events: 3 [ 0.054014] ... event mask: 000000070000000f [ 0.055290] rcu: Hierarchical SRCU implementation. [ 0.057281] smp: Bringing up secondary CPUs ... [ 0.058576] x86: Booting SMP configuration: [ 0.059044] .... node #0, CPUs: #1 #2 #3 [ 0.064124] smp: Brought up 1 node, 4 CPUs [ 0.066014] smpboot: Max logical packages: 1 [ 0.067014] smpboot: Total of 4 processors activated (19199.98 BogoMIPS) [ 0.130924] node 0 deferred pages initialised in 63ms [ 0.135378] devtmpfs: initialized [ 0.136244] x86/mm: Memory block size: 128MB [ 0.139000] gcov: version magic: 0x41383552 [ 0.143468] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.148079] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.151277] pinctrl core: initialized pinctrl subsystem [ 0.154151] [ 0.154861] ************************************************************* [ 0.157014] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.160012] ** ** [ 0.162010] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.165012] ** ** [ 0.168012] ** This means that this kernel is built to expose internal ** [ 0.171014] ** IOMMU data structures, which may compromise security on ** [ 0.173007] ** your system. ** [ 0.174009] ** ** [ 0.176010] ** If you see this message and you are not debugging the ** [ 0.179011] ** kernel, report this immediately to your vendor! ** [ 0.182014] ** ** [ 0.184009] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.186011] ************************************************************* [ 0.189121] NET: Registered protocol family 16 [ 0.191482] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.194054] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.196059] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.200145] cpuidle: using governor menu [ 0.201558] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.204611] PCI: Using configuration type 1 for base access [ 0.207121] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.216371] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.218016] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.222145] cryptd: max_cpu_qlen set to 1000 [ 0.225129] ACPI: Added _OSI(Module Device) [ 0.226013] ACPI: Added _OSI(Processor Device) [ 0.228012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.230021] ACPI: Added _OSI(Processor Aggregator Device) [ 0.235263] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.238501] ACPI: Interpreter enabled [ 0.240116] ACPI: PM: (supports S0 S3 S4 S5) [ 0.242014] ACPI: Using IOAPIC for interrupt routing [ 0.243122] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.247562] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.257637] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.259039] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.262023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.266078] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.271350] acpiphp: Slot [2] registered [ 0.273118] acpiphp: Slot [5] registered [ 0.274447] acpiphp: Slot [6] registered [ 0.276308] acpiphp: Slot [7] registered [ 0.278184] acpiphp: Slot [8] registered [ 0.280301] acpiphp: Slot [9] registered [ 0.281197] acpiphp: Slot [10] registered [ 0.286253] acpiphp: Slot [3] registered [ 0.288150] acpiphp: Slot [4] registered [ 0.290485] acpiphp: Slot [11] registered [ 0.293172] acpiphp: Slot [12] registered [ 0.295147] acpiphp: Slot [13] registered [ 0.297195] acpiphp: Slot [14] registered [ 0.299150] acpiphp: Slot [15] registered [ 0.301108] acpiphp: Slot [16] registered [ 0.302110] acpiphp: Slot [17] registered [ 0.304135] acpiphp: Slot [18] registered [ 0.305096] acpiphp: Slot [19] registered [ 0.307124] acpiphp: Slot [20] registered [ 0.309245] acpiphp: Slot [21] registered [ 0.310086] acpiphp: Slot [22] registered [ 0.312164] acpiphp: Slot [23] registered [ 0.314109] acpiphp: Slot [24] registered [ 0.315092] acpiphp: Slot [25] registered [ 0.317111] acpiphp: Slot [26] registered [ 0.319198] acpiphp: Slot [27] registered [ 0.320102] acpiphp: Slot [28] registered [ 0.322082] acpiphp: Slot [29] registered [ 0.323090] acpiphp: Slot [30] registered [ 0.324117] acpiphp: Slot [31] registered [ 0.326147] PCI host bridge to bus 0000:00 [ 0.328022] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.331038] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.333035] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.336040] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.338032] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.341037] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.343331] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.347247] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.351000] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.359912] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.366197] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.369027] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.371025] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.374026] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.376304] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.379943] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.383048] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.386012] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.392147] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.404015] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.413020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.420433] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.429024] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.434022] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.452019] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.467196] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.476024] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.487021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.505417] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.515915] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.532021] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.543034] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.572025] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.584813] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.591015] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.601019] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.619025] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.633154] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.643015] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.650013] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.676022] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.695193] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.707015] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.714016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.730021] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.743875] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.746469] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.749512] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.752448] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.755302] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.760182] iommu: Default domain type: Passthrough [ 0.764083] SCSI subsystem initialized [ 0.766104] ACPI: bus type USB registered [ 0.767180] usbcore: registered new interface driver usbfs [ 0.769070] usbcore: registered new interface driver hub [ 0.771155] usbcore: registered new device driver usb [ 0.773151] pps_core: LinuxPPS API ver. 1 registered [ 0.774009] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.777052] PTP clock support registered [ 0.780068] EDAC MC: Ver: 3.0.0 [ 0.782580] PCI: Using ACPI for IRQ routing [ 0.785628] NetLabel: Initializing [ 0.788017] NetLabel: domain hash size = 128 [ 0.790012] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.793081] NetLabel: unlabeled traffic allowed by default [ 0.796140] vgaarb: loaded [ 0.797296] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.800022] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.806374] clocksource: Switched to clocksource kvm-clock [ 0.957534] VFS: Disk quotas dquot_6.6.0 [ 0.959305] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.964093] *** VALIDATE ramfs *** [ 0.965123] *** VALIDATE hugetlbfs *** [ 0.968445] pnp: PnP ACPI init [ 0.971544] pnp: PnP ACPI: found 6 devices [ 0.997342] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.001160] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.003606] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.006046] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.008804] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.010867] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.013888] NET: Registered protocol family 2 [ 1.016634] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.022993] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.027591] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.034728] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.038774] TCP: Hash tables configured (established 65536 bind 65536) [ 1.042341] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.046765] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.050870] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.054912] NET: Registered protocol family 1 [ 1.061314] RPC: Registered named UNIX socket transport module. [ 1.063836] RPC: Registered udp transport module. [ 1.065816] RPC: Registered tcp transport module. [ 1.067732] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.070572] NET: Registered protocol family 44 [ 1.073800] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.076711] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.079716] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.082357] PCI: CLS 0 bytes, default 64 [ 1.084101] Unpacking initramfs... [ 2.554100] debug: unmapping init [mem 0xffff8b40bcc54000-0xffff8b40bffbffff] [ 2.558307] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.560629] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.563785] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229835b7123, max_idle_ns: 440795242976 ns [ 3.151943] Initialise system trusted keyrings [ 3.153766] Key type blacklist registered [ 3.156167] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.171550] zbud: loaded [ 3.176784] *** VALIDATE nfs *** [ 3.178168] *** VALIDATE nfs4 *** [ 3.180410] pstore: using deflate compression [ 3.187596] Platform Keyring initialized [ 3.307074] NET: Registered protocol family 38 [ 3.308795] Key type asymmetric registered [ 3.310109] Asymmetric key parser 'x509' registered [ 3.311804] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.314581] io scheduler mq-deadline registered [ 3.316440] io scheduler kyber registered [ 3.318124] io scheduler bfq registered [ 3.320284] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.324261] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.327492] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.331117] ACPI: Power Button [PWRF] [ 3.337623] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.345405] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.365753] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.385766] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.408247] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.439299] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.471090] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.478217] Non-volatile memory driver v1.3 [ 3.479900] Linux agpgart interface v0.103 [ 3.515108] virtio_blk virtio1: [vda] 146672 512-byte logical blocks (75.1 MB/71.6 MiB) [ 3.518245] vda: detected capacity change from 0 to 75096064 [ 3.536538] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.539288] vdb: detected capacity change from 0 to 1073741824 [ 3.555082] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.557779] vdc: detected capacity change from 0 to 2621440000 [ 3.577606] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.580282] vdd: detected capacity change from 0 to 2621440000 [ 3.595423] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.597993] vde: detected capacity change from 0 to 4294967296 [ 3.616925] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.619778] vdf: detected capacity change from 0 to 4294967296 [ 3.627059] libphy: Fixed MDIO Bus: probed [ 3.638747] usbcore: registered new interface driver usbserial_generic [ 3.641723] usbserial: USB Serial support registered for generic [ 3.644803] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.651961] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.654302] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.659365] mousedev: PS/2 mouse device common for all mice [ 3.664701] rtc_cmos 00:05: RTC can wake from S4 [ 3.668206] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.672573] rtc_cmos 00:05: registered as rtc0 [ 3.672653] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.672697] intel_pstate: CPU model not supported [ 3.677901] hid: raw HID events driver (C) Jiri Kosina [ 3.686807] usbcore: registered new interface driver usbhid [ 3.688755] usbhid: USB HID core driver [ 3.690477] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.693312] drop_monitor: Initializing network drop monitor service [ 3.702126] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.702687] Initializing XFRM netlink socket [ 3.708267] NET: Registered protocol family 10 [ 3.711486] Segment Routing with IPv6 [ 3.712994] NET: Registered protocol family 17 [ 3.714981] mpls_gso: MPLS GSO support [ 3.721232] RAS: Correctable Errors collector initialized. [ 3.723418] AVX version of gcm_enc/dec engaged. [ 3.725031] AES CTR mode by8 optimization enabled [ 3.812380] sched_clock: Marking stable (3812350699, 0)->(4735876096, -923525397) [ 3.815948] registered taskstats version 1 [ 3.818466] Loading compiled-in X.509 certificates [ 3.820685] zswap: loaded using pool lzo/zbud [ 3.851346] Key type big_key registered [ 3.867188] Key type encrypted registered [ 3.868936] ima: No TPM chip found, activating TPM-bypass! [ 3.871066] ima: Allocated hash algorithm: sha1 [ 3.873223] ima: No architecture policies found [ 3.875044] evm: Initialising EVM extended attributes: [ 3.877282] evm: security.selinux [ 3.878854] evm: security.ima [ 3.880080] evm: security.capability [ 3.881749] evm: HMAC attrs: 0x1 [ 3.884748] rtc_cmos 00:05: setting system clock to 2026-08-27 10:50:03 UTC (1787827803) [ 3.892722] debug: unmapping init [mem 0xffffffff97a03000-0xffffffff97bfffff] [ 3.898298] debug: unmapping init [mem 0xffffffff96782000-0xffffffff96a58fff] [ 3.908078] Write protecting the kernel read-only data: 28672k [ 3.911447] debug: unmapping init [mem 0xffffffff94e03000-0xffffffff94ffffff] [ 3.914850] debug: unmapping init [mem 0xffffffff95714000-0xffffffff957fffff] [ 3.959062] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.967549] systemd[1]: Detected virtualization kvm. [ 3.969539] systemd[1]: Detected architecture x86-64. [ 3.971523] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 4.001974] systemd[1]: No hostname configured. [ 4.003468] systemd[1]: Set hostname to . [ 4.005340] random: systemd: uninitialized urandom read (16 bytes read) [ 4.006978] systemd[1]: Initializing machine ID from random generator. [ 4.053925] random: ln: uninitialized urandom read (6 bytes read) [ 4.166182] random: systemd: uninitialized urandom read (16 bytes read) [ 4.170438] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ 4.180262] systemd[1]: Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket (/dev/log). [ 4.190290] systemd[1]: Reached target Timers. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Local File Systems. [ OK ] Reached target Swap. [ OK ] Listening on Journal Socket. Starting Journal Service... Starting Create Volatile Files and Directories... Starting Setup Virtual Console... Starting Apply Kernel Variables... [ OK ] Started Memstrack Anylazing Service. Starting Create list of required st…ce nodes for the current kernel... [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Slices. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. [ 4.852940] random: fast init done Starting dracut pre-udev hook... [ 4.909664] device-mapper: uevent: version 1.0.3 [ 4.912159] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.765848] virtio_net virtio0 ens2: renamed from eth0 [ 5.823586] scsi host0: ata_piix [ 5.861842] scsi host1: ata_piix [ 5.865708] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.869880] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.731420] dracut-initqueue[589]: RTNETLINK answers: File exists [ 10.388282] random: crng init done [ 10.389661] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 11.010918] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Paths. [ OK ] Stopped target Slices. [ OK ] Stopped target System Initialization. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. Stopping udev Kernel Device Manager... [ OK ] Stopped target Local File Systems. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped target Swap. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.188449] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.466042] SELinux: Disabled at runtime. [ 12.527945] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.538198] systemd[1]: Detected virtualization kvm. [ 12.540879] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.068105] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.073828] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.079247] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.084318] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.087601] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.094643] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.106138] systemd[1]: Listening on RPCbind Server Activation Socket. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Created slice system-sshd\x2dkeygen.slice. Mounting Kernel Debug File System... [ OK ] Created slice system-getty.slice. [ OK ] Reached target rpc_pipefs.target. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Reached target RPC Port Mapper. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd Root File System. Mounting POSIX Message Queue File System... Mounting Huge Pages File System... [ 13.185733] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Listening on udev Kernel Socket. Starting Remount Root and Kernel File Systems... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. Starting Apply Kernel Variables... [ OK ] Stopped target Initrd File Systems. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Listening on initctl Compatibility Named Pipe. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Started Apply Kernel Variables. Starting Create Static Device Nodes in /dev... Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.577533] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 13.958411] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 13.965699] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.170848] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.186252] EDAC sbridge: Ver: 1.1.2 [ 15.872466] Key type dns_resolver registered [ 16.202911] NFS: Registering the id_resolver key type [ 16.206170] Key type id_resolver registered [ 16.208119] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Rebuild Dynamic Linker Cache... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... Starting Login Service... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Started irqbalance daemon. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. Starting Network Manager... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... Starting Network Manager Wait Online... Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg656-server login: [ 38.313060] hrtimer: interrupt took 3081024 ns [ 66.217560] libcfs: loading out-of-tree module taints kernel. [ 66.348131] Key type ._llcrypt registered [ 66.350991] Key type .llcrypt registered [ 66.482965] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 88.446229] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 90.153508] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 90.168987] alg: No test for adler32 (adler32-zlib) [ 91.608827] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 92.569366] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 94.304156] Key type lgssc registered [ 96.361483] Lustre: Echo OBD driver; http://www.lustre.org/ [ 112.538171] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 150.363323] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 164.077164] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 164.139436] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 165.416406] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 165.459474] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 165.597462] Lustre: lustre-MDT0000: new disk, initializing [ 165.729189] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 165.762143] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 170.701684] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 184.626725] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 184.696221] Lustre: 6528:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 184.747201] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 184.754973] Lustre: Skipped 1 previous similar message [ 184.876508] Lustre: lustre-MDT0001: new disk, initializing [ 184.967536] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 185.003371] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 185.014778] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 189.463391] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 194.379272] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 203.755464] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 204.067703] Lustre: lustre-OST0000: new disk, initializing [ 204.073107] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 204.081767] Lustre: 8467:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 204.183730] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 205.502101] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 205.528164] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 205.584567] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 210.185899] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 225.136303] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 225.294992] Lustre: lustre-OST0001: new disk, initializing [ 225.299559] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 225.317896] Lustre: 9540:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 225.388289] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 231.697581] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 232.009593] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 232.018239] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 232.076754] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 243.147649] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 248.897469] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 256.343963] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing check_logdir /tmp/testlogs/ [ 261.942410] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing yml_node [ 265.977923] Lustre: DEBUG MARKER: Client: 2.17.57.65 [ 268.671000] Lustre: DEBUG MARKER: MDS: 2.17.57.65 [ 271.415409] Lustre: DEBUG MARKER: OSS: 2.17.57.65 [ 273.197937] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Thu Aug 27 06:54:31 EDT 2026 [ 291.000670] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 292.967862] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 313.313314] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 313.330735] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 313.356231] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.338893] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 314.341545] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 316.109855] Lustre: server umount lustre-MDT0000 complete [ 319.927873] LustreError: 7478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828119 with bad export cookie 8388681923326086884 [ 319.928048] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 319.936344] LustreError: 7478:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 324.578106] LustreError: 9319:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 324.580761] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 324.582551] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 324.582558] Lustre: Skipped 3 previous similar messages [ 324.603229] LustreError: 9319:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 324.636827] Lustre: Skipped 3 previous similar messages [ 329.697207] LustreError: 6536:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 329.697795] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 329.733203] LustreError: 6536:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 329.768224] Lustre: Skipped 2 previous similar messages [ 330.722145] LustreError: 6538:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 330.746225] LustreError: 6538:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 334.307423] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 334.592121] Lustre: server umount lustre-MDT0001 complete [ 338.427616] Lustre: server umount lustre-OST0000 complete [ 342.837855] Lustre: server umount lustre-OST0001 complete [ 350.538219] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 360.613335] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 395.042216] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 395.361584] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 395.410310] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 395.528518] Lustre: lustre-MDT0000: new disk, initializing [ 395.644651] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 395.678930] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 400.564445] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 411.050417] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 411.179377] Lustre: 15093:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 411.261265] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 411.265316] Lustre: Skipped 1 previous similar message [ 411.336379] Lustre: lustre-MDT0001: new disk, initializing [ 411.429188] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 411.463824] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 411.472035] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 415.705358] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 425.184396] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 430.876909] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 437.735653] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 438.069393] Lustre: lustre-OST0000: new disk, initializing [ 438.075569] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 438.083272] Lustre: 17061:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 438.192923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 439.311529] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 439.333195] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 439.495496] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 444.356834] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 454.070914] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 459.159246] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 459.325800] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 463.726239] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 463.932382] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 470.498275] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 470.506194] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 470.510270] LustreError: Skipped 1 previous similar message [ 470.527720] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 470.538803] Lustre: Skipped 3 previous similar messages [ 480.224281] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 480.350028] Lustre: server umount lustre-OST0000 complete [ 485.859453] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 485.863400] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 485.883892] Lustre: Skipped 2 previous similar messages [ 485.901510] Lustre: Skipped 3 previous similar messages [ 498.145188] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 498.411821] Lustre: server umount lustre-MDT0000 complete [ 501.224433] LustreError: 15100:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 501.246969] LustreError: 15100:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 501.976967] LustreError: 15084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828301 with bad export cookie 8388681923326088417 [ 501.980738] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 501.997490] LustreError: 15084:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 502.595505] Lustre: server umount lustre-MDT0001 complete [ 507.553783] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 06:58:25 (1787828305) [ 514.168147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 514.531414] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 514.592494] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 518.083374] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 524.771828] LustreError: 19331:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 524.801259] LustreError: 19331:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 525.230757] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 529.648645] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 536.503812] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 542.185489] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 549.041171] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 555.626661] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 562.383238] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 565.620688] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 571.883110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 571.895569] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 571.917860] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 571.928402] Lustre: Skipped 4 previous similar messages [ 577.025695] Lustre: server umount lustre-MDT0000 complete [ 580.073884] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 580.091206] LustreError: 19330:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 580.386498] LustreError: 19311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828380 with bad export cookie 8388681923326089299 [ 580.391350] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 580.398025] LustreError: 19311:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 580.722569] Lustre: server umount lustre-MDT0001 complete [ 594.298219] Lustre: server umount lustre-OST0000 complete [ 613.009416] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 622.026827] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 622.349888] LustreError: 23707:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 622.404995] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 622.409388] Lustre: Skipped 2 previous similar messages [ 626.188663] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 633.849511] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 637.711792] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 639.791674] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 645.650766] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 652.906772] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 660.896711] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 661.015381] Lustre: lustre-OST0001: new disk, initializing [ 661.019790] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 661.028443] Lustre: 26022:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 662.685308] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 662.705510] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 662.940043] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 668.532966] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 679.711233] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 684.096560] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 695.784627] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 695.793912] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 695.809085] Lustre: Skipped 2 previous similar messages [ 695.822943] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 695.827824] Lustre: Skipped 3 previous similar messages [ 701.924606] Lustre: server umount lustre-MDT0000 complete [ 703.972241] LustreError: 23703:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 703.997763] LustreError: 23703:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 705.746560] LustreError: 24452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828505 with bad export cookie 8388681923326090146 [ 705.752050] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 705.759309] LustreError: 24452:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 706.163415] Lustre: server umount lustre-MDT0001 complete [ 720.337634] Lustre: server umount lustre-OST0000 complete [ 734.476579] Lustre: server umount lustre-OST0001 complete [ 749.986938] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 759.198730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 759.600115] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 759.611113] Lustre: Skipped 3 previous similar messages [ 764.504993] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 773.204135] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 778.349366] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 781.645205] Lustre: 30208:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 788.013085] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 794.613751] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 803.856318] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 810.672757] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 818.994573] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 822.600377] Lustre: 32087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 829.922550] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 829.932208] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 829.946617] Lustre: Skipped 3 previous similar messages [ 829.951344] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 829.954683] Lustre: Skipped 4 previous similar messages [ 834.568741] Lustre: server umount lustre-MDT0000 complete [ 836.071253] LustreError: 29054:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 836.082912] LustreError: 29054:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 8 previous similar messages [ 837.821181] LustreError: 29039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828637 with bad export cookie 8388681923326091441 [ 837.825677] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 837.832049] LustreError: 29039:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 838.164982] Lustre: server umount lustre-MDT0001 complete [ 852.266493] Lustre: server umount lustre-OST0000 complete [ 866.392312] Lustre: server umount lustre-OST0001 complete [ 874.942196] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 07:04:32 (1787828672) [ 902.054571] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 912.936161] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 917.897872] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 927.591596] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 933.018788] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 935.989993] Lustre: 36040:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 942.018797] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 948.313492] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 958.138694] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 965.607789] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 973.260416] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 976.971488] Lustre: 37918:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 984.560160] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 984.565559] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 984.573877] Lustre: Skipped 3 previous similar messages [ 984.574854] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 984.574861] Lustre: Skipped 4 previous similar messages [ 988.557641] Lustre: server umount lustre-MDT0000 complete [ 992.090585] LustreError: 34874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828791 with bad export cookie 8388681923326092547 [ 992.091986] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 992.105104] LustreError: 34874:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 992.472514] Lustre: server umount lustre-MDT0001 complete [ 1005.501294] Lustre: server umount lustre-OST0000 complete [ 1019.105596] Lustre: server umount lustre-OST0001 complete [ 1026.752666] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 07:07:04 (1787828824) [ 1044.158654] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1054.199387] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1054.652031] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1054.666446] Lustre: Skipped 7 previous similar messages [ 1058.786965] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1066.832839] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1071.689898] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1074.633237] Lustre: 41211:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1081.780749] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1089.296054] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1092.393569] LustreError: 41566:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1092.415344] LustreError: 41566:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 14 previous similar messages [ 1098.513807] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1104.707112] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1112.155255] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1115.700636] Lustre: 43096:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1116.480813] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1116.496805] Lustre: Skipped 1 previous similar message [ 1120.226267] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1120.248532] Lustre: Skipped 6 previous similar messages [ 1125.463838] Lustre: server umount lustre-MDT0000 complete [ 1129.907166] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787828929 with bad export cookie 8388681923326093653 [ 1129.911899] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1129.920533] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1144.288691] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1144.714125] Lustre: server umount lustre-MDT0001 complete [ 1153.580408] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1154.721127] LustreError: 43956:0:(import.c:339:ptlrpc_invalidate_import()) MGS: timeout waiting for callback (1 != 0) [ 1154.734543] LustreError: 43956:0:(import.c:363:ptlrpc_invalidate_import()) @@@ still on sending list req@ffff8b410408b0c0 x1874673420077568/t0(0) o250->MGC192.168.206.156@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 1787828954 ref 1 fl Rpc:NQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1154.771849] LustreError: 43956:0:(import.c:373:ptlrpc_invalidate_import()) MGS: Unregistering RPCs found (0). Network is sluggish? Waiting for them to error out. [ 1155.045527] LustreError: 3662:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff8b4005b1f480 x1874673420078720/t0(0) o250->MGC192.168.206.156@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1160.173738] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1168.244886] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1169.839495] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1173.412995] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1180.457992] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1181.711908] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1186.491186] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1188.253457] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1194.466911] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1199.091423] Lustre: server umount lustre-MDT0000 complete [ 1202.909354] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787829002 with bad export cookie 8388681923326094752 [ 1202.914050] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1202.926862] LustreError: 40041:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1203.369841] Lustre: server umount lustre-MDT0001 complete [ 1217.500262] Lustre: server umount lustre-OST0000 complete [ 1231.882196] Lustre: server umount lustre-OST0001 complete [ 1242.728548] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 07:10:39 (1787829039) [ 1260.662676] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1271.812925] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1277.671599] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1286.710514] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1291.858978] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1295.081953] Lustre: 48852:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1302.164496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1310.276816] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1322.034303] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1331.053682] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1339.547973] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1344.850714] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1358.312269] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1358.326964] Lustre: Skipped 30 previous similar messages [ 1363.531952] Lustre: server umount lustre-MDT0000 complete [ 1367.597603] LustreError: 47681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787829167 with bad export cookie 8388681923326095571 [ 1367.610223] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1367.615252] LustreError: 47681:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1367.951480] Lustre: server umount lustre-MDT0001 complete [ 1381.630727] Lustre: server umount lustre-OST0000 complete [ 1384.544292] Lustre: 3663:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787829168/real 1787829168] req@ffff8b410a4b52c0 x1874673420171264/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787829184 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1384.586196] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1384.601570] Lustre: Skipped 10 previous similar messages [ 1386.031972] Lustre: server umount lustre-OST0001 complete [ 1397.177915] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 07:13:14 (1787829194) [ 1400.482057] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1402.746759] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 07:13:20 (1787829200) [ 1415.156352] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 1423.839874] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1446.372093] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1446.702780] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1446.757432] Lustre: lustre-MDT0000: new disk, initializing [ 1446.849436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1451.377436] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1461.629045] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1468.638217] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1468.960556] Lustre: lustre-OST0000: new disk, initializing [ 1468.963642] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1468.969216] Lustre: Skipped 1 previous similar message [ 1468.973612] Lustre: 54941:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1471.089239] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1471.112142] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1471.176608] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1475.486479] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1485.587293] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1487.903532] Lustre: DEBUG MARKER: create test files [ 1522.145239] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1526.783278] Lustre: server umount lustre-OST0000 complete [ 1531.343712] Lustre: server umount lustre-MDT0000 complete [ 1543.100517] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 1546.478938] Key type lgssc unregistered [ 1546.852861] LNet: 56575:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1546.858294] LNetError: 56575:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1546.885204] LNet: Removed LNI 192.168.206.156@tcp [ 1548.086168] Key type .llcrypt unregistered [ 1548.088963] Key type ._llcrypt unregistered [ 1576.129991] Key type ._llcrypt registered [ 1576.133852] Key type .llcrypt registered [ 1576.271801] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1577.497995] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1577.537416] alg: No test for adler32 (adler32-zlib) [ 1578.574412] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 1578.844979] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 1580.496142] Key type lgssc registered [ 1581.541989] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1589.074908] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1589.128082] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1590.826869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1595.557851] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1603.553696] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1611.059752] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1611.583246] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1615.668438] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 1615.820367] Lustre: 59286:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 1615.841915] Lustre: 59286:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 1615.856072] Lustre: 59286:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1615.869810] Lustre: 59286:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1615.876549] Lustre: 59286:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1615.882097] Lustre: 59286:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1618.968740] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1626.706682] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1633.260771] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1639.384449] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1649.307972] Lustre: server umount lustre-OST0000 complete [ 1662.675249] Lustre: server umount lustre-MDT0000 complete [ 1675.710914] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 1679.229964] Key type lgssc unregistered [ 1679.618773] LNet: 60901:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1679.633173] LNetError: 60901:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1679.675875] LNet: Removed LNI 192.168.206.156@tcp [ 1680.637554] Key type .llcrypt unregistered [ 1680.639235] Key type ._llcrypt unregistered [ 1714.650035] Key type ._llcrypt registered [ 1714.651871] Key type .llcrypt registered [ 1714.763979] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1715.707319] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1715.743540] alg: No test for adler32 (adler32-zlib) [ 1716.912592] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 1717.194583] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 1718.904179] Key type lgssc registered [ 1720.276285] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1728.260818] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1728.290040] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1729.903418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1735.090233] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1743.420787] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1751.507696] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1752.136535] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1755.260543] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1755.331134] Lustre: 63738:0:(osd_internal.h:1460:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1755.338646] Lustre: 63738:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1755.344650] Lustre: 63738:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1755.356345] Lustre: 63738:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1755.362638] Lustre: 63738:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1755.368777] Lustre: 63738:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1759.264592] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1767.316815] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1772.801939] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1786.339750] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1786.354689] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1786.374314] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1788.388607] Lustre: server umount lustre-OST0000 complete [ 1792.480869] Lustre: server umount lustre-MDT0000 complete [ 1805.808156] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 1808.993234] Key type lgssc unregistered [ 1809.333599] LNet: 65298:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1809.342851] LNetError: 65298:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1809.358814] LNet: Removed LNI 192.168.206.156@tcp [ 1810.213311] Key type .llcrypt unregistered [ 1810.215752] Key type ._llcrypt unregistered [ 1832.819988] Key type ._llcrypt registered [ 1832.827406] Key type .llcrypt registered [ 1833.015707] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 1849.892589] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 1851.036030] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1851.054468] alg: No test for adler32 (adler32-zlib) [ 1852.224333] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 1852.591263] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 1854.248180] Key type lgssc registered [ 1855.719467] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1896.489467] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1896.516396] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1897.776689] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1897.802945] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1897.911757] Lustre: lustre-MDT0000: new disk, initializing [ 1898.013903] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1898.049206] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1902.086611] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1914.121041] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1914.360990] Lustre: 68454:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1914.419074] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1914.425228] Lustre: Skipped 1 previous similar message [ 1914.508498] Lustre: lustre-MDT0001: new disk, initializing [ 1914.576116] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1914.624687] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1914.637442] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1919.338426] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1930.324071] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1936.858622] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1943.220570] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1943.597325] Lustre: lustre-OST0000: new disk, initializing [ 1943.600695] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1943.605676] Lustre: 70422:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1943.676881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1949.350368] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1949.358134] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1949.583918] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1949.935865] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 1960.744840] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1965.507849] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1965.721701] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1970.857856] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1971.113698] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1975.277894] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1975.285683] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1975.298155] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1975.359439] Lustre: Skipped 1 previous similar message [ 1979.725665] Lustre: server umount lustre-OST0000 complete [ 1985.505433] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1985.520934] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1985.522539] Lustre: Skipped 1 previous similar message [ 1985.531997] Lustre: Skipped 1 previous similar message [ 1990.414071] Lustre: server umount lustre-MDT0000 complete [ 1994.207845] LustreError: 68446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787829793 with bad export cookie 4812017307071030757 [ 1994.231375] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1994.233763] LustreError: 68446:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1994.726425] Lustre: server umount lustre-MDT0001 complete [ 2003.998666] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 07:23:21 (1787829801) [ 2014.849958] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 2056.590215] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 2067.539123] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 2105.010153] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2105.283831] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2105.331804] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2105.445725] Lustre: lustre-MDT0000: new disk, initializing [ 2105.503148] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2105.520390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2109.900936] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2120.093825] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2120.202335] Lustre: 76456:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2120.245199] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2120.248483] Lustre: Skipped 1 previous similar message [ 2120.304785] Lustre: lustre-MDT0001: new disk, initializing [ 2120.378075] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2120.421807] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2120.428173] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2124.931506] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2134.163120] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2139.777260] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2146.679197] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2146.915652] Lustre: lustre-OST0000: new disk, initializing [ 2146.917490] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2146.924274] Lustre: 78425:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2147.001309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2153.311492] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2154.587643] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2154.603678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2154.726646] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2163.592159] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2168.515192] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2168.721980] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2173.880215] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2174.062313] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2180.578043] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2180.589782] LustreError: Skipped 1 previous similar message [ 2180.595048] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2180.614462] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2180.617187] Lustre: Skipped 1 previous similar message [ 2180.634440] Lustre: Skipped 1 previous similar message [ 2185.701539] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2185.710567] Lustre: Skipped 1 previous similar message [ 2190.304284] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2190.415803] Lustre: server umount lustre-OST0000 complete [ 2195.942789] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2195.947804] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2195.953673] Lustre: Skipped 1 previous similar message [ 2195.961545] Lustre: Skipped 1 previous similar message [ 2200.827774] Lustre: server umount lustre-MDT0000 complete [ 2201.068504] LustreError: 76469:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2201.101077] LustreError: 76469:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [ 2204.447169] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787830004 with bad export cookie 4812017307071031639 [ 2204.459756] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2204.462794] LustreError: 76448:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2204.959371] Lustre: server umount lustre-MDT0001 complete [ 2214.796570] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 07:26:52 (1787830012) [ 2221.532343] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2222.038586] LustreError: 80843:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2222.112100] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2227.155440] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2227.178081] LustreError: 80844:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2232.291297] LustreError: 80843:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2235.682251] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2241.033237] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2248.326228] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2253.685470] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2260.446118] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2260.870992] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2260.875506] Lustre: Skipped 1 previous similar message [ 2267.796182] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2275.157152] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2278.588694] Lustre: 83514:0:(genops.c:1774:obd_export_evict_by_uuid()) MGS: evicting b1a90914-5dc7-43ef-8b12-9130bb2202ce at adminstrative request [ 2282.481196] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2282.512509] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2292.712312] Lustre: MGS: Client b1a90914-5dc7-43ef-8b12-9130bb2202ce (at 0@lo) reconnecting [ 2292.726950] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x42c7b83ee1002cc9 to 0x42c7b83ee1002f69 [ 2292.748602] Lustre: MGC192.168.206.156@tcp: Connection restored to 0@lo (at 0@lo) [ 2315.714759] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2315.922528] Lustre: lustre-OST0001: new disk, initializing [ 2315.925759] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2315.935087] Lustre: 83907:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2323.443631] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2325.069160] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2325.094514] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2325.246219] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2334.270908] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2340.838699] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2340.839685] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2340.849666] LustreError: Skipped 2 previous similar messages [ 2340.854037] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2340.930021] Lustre: Skipped 1 previous similar message [ 2343.469041] Lustre: server umount lustre-OST0001 complete [ 2345.960701] LustreError: 82665:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2345.987919] LustreError: 82665:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2349.026147] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2349.038196] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2349.041000] LustreError: Skipped 1 previous similar message [ 2349.079400] Lustre: Skipped 1 previous similar message [ 2353.895085] Lustre: server umount lustre-OST0000 complete [ 2359.269913] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2359.283253] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2359.291225] Lustre: Skipped 5 previous similar messages [ 2364.396477] Lustre: server umount lustre-MDT0000 complete [ 2366.433701] LustreError: 82683:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2366.457518] LustreError: 82683:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2368.226782] LustreError: 80823:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787830167 with bad export cookie 4812017307071033193 [ 2368.243157] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2368.652961] Lustre: server umount lustre-MDT0001 complete [ 2377.941215] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 07:29:35 (1787830175) [ 2379.741776] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2381.750408] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 07:29:39 (1787830179) [ 2383.228179] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2384.945864] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 07:29:42 (1787830182) [ 2387.007877] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2388.876129] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 07:29:46 (1787830186) [ 2391.209886] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2393.986767] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 07:29:50 (1787830190) [ 2396.582752] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2404.552572] loop: module loaded [ 2404.587042] loop0: detected capacity change from 0 to 4294967296 [ 2404.706809] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2407.391501] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2411.751452] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2416.350285] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2442.931059] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2442.973976] LustreError: 88031:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2442.987748] LustreError: 88031:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2442.992753] LustreError: 88031:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2442.999054] LustreError: 88031:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2462.536492] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 2473.722037] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 2512.001122] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2512.233270] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2512.258388] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2512.334392] Lustre: lustre-MDT0000: new disk, initializing [ 2512.449286] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2512.460851] Lustre: Skipped 1 previous similar message [ 2512.514597] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2517.144481] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2528.396107] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2528.498443] Lustre: 90897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2528.820413] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2533.169382] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2543.088657] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2550.060760] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2557.075261] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2557.424433] Lustre: 92872:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2564.471414] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2565.194725] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2565.336441] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2574.626915] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2579.356629] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2579.706642] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2583.621519] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2583.825658] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2586.101233] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2586.103311] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2586.139219] Lustre: Skipped 2 previous similar messages [ 2586.150803] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2586.153455] Lustre: Skipped 2 previous similar messages [ 2591.810666] Lustre: server umount lustre-OST0000 complete [ 2602.448904] Lustre: server umount lustre-MDT0000 complete [ 2606.573703] LustreError: 90905:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2606.617306] LustreError: 90905:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 2606.847744] LustreError: 91693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787830406 with bad export cookie 4812017307071033809 [ 2606.852245] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2606.865656] LustreError: 91693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 8 previous similar messages [ 2607.256717] Lustre: server umount lustre-MDT0001 complete [ 2619.867401] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 07:33:37 (1787830417) [ 2628.423329] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2634.028540] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2639.330554] LustreError: 95282:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2639.342328] LustreError: 95282:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 2643.883518] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2644.240185] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2644.243388] Lustre: Skipped 3 previous similar messages [ 2649.469496] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2660.585303] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2667.855103] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2676.511033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2682.462118] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2689.638636] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2693.543409] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2700.183314] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 6 sec [ 2704.933077] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2705.163740] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2711.551264] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2711.727508] Lustre: lustre-OST0001: new disk, initializing [ 2711.731374] Lustre: Skipped 2 previous similar messages [ 2711.735500] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2711.741150] Lustre: Skipped 3 previous similar messages [ 2711.746337] Lustre: 98697:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2718.567864] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2719.861629] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2719.877489] Lustre: Skipped 2 previous similar messages [ 2719.884233] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2719.981766] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2727.840538] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2732.531718] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2732.814553] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2737.375530] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2737.626337] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2757.432612] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2783.717066] Lustre: Failing over lustre-MDT0000 [ 2784.193697] Lustre: server umount lustre-MDT0000 complete [ 2786.811690] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2786.829656] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2786.842055] Lustre: Skipped 5 previous similar messages [ 2786.869475] LustreError: 95278:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 2803.168128] Lustre: 66676:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787830586/real 1787830586] req@00000000f7021ef7 x1874675265448704/t0(0) o400->MGC192.168.206.156@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1787830602 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2803.198360] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2808.987510] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2808.990818] LDISKFS-fs (dm-0): recovery complete [ 2809.001504] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2812.392606] LustreError: 66674:0:(client.c:1404:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000e052d3c2 x1874675265456512/t0(0) o250->MGC192.168.206.156@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2812.762175] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2813.482955] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2813.484318] Lustre: 99339:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2817.153154] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 2818.028143] Lustre: 96011:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2818.038560] Lustre: 96011:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 1 previous similar message [ 2818.062862] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2818.131411] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2818.448126] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2818.775287] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2818.785701] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2819.112310] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2819.120789] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 1 previous similar message [ 2819.144982] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2819.158979] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 7 previous similar messages [ 2819.802910] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2819.808360] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2820.128735] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2820.132972] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 2 previous similar messages [ 2821.417915] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2821.431839] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 6 previous similar messages [ 2822.074761] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2822.090712] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2822.416105] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2822.428430] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 6 previous similar messages [ 2825.627685] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2825.642391] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 12 previous similar messages [ 2826.298637] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2826.312563] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2826.632735] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2826.641756] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 12 previous similar messages [ 2833.716245] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2833.734824] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 24 previous similar messages [ 2834.378645] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2834.392795] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2834.728149] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2834.740853] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 24 previous similar messages [ 2849.913491] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2849.939388] Lustre: 101546:0:(ldlm_lib.c:2110:extend_recovery_timer()) Skipped 50 previous similar messages [ 2850.582131] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2850.585948] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2850.912372] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2850.921448] LustreError: 101546:0:(ldlm_lib.c:2723:replay_request_or_update()) Skipped 50 previous similar messages [ 2873.088157] Lustre: 101546:0:(ldlm_lib.c:2419:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 2873.094163] Lustre: 101546:0:(ldlm_lib.c:2429:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 2873.106560] Lustre: 101546:0:(genops.c:1601:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 859b3993-e1a0-4abd-85d4-455acfbc6fc7@192.168.206.56@tcp [ 2873.117224] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 2873.124482] Lustre: 101546:0:(ldlm_lib.c:1939:abort_req_replay_queue()) @@@ aborted: req@00000000ec264035 x1874675247418112/t0(8589934769) o101->859b3993-e1a0-4abd-85d4-455acfbc6fc7@192.168.206.56@tcp:498/0 lens 592/0 e 0 to 0 dl 1787830683 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 2873.149766] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 2873.163864] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 2873.179019] Lustre: lustre-MDT0000: Denying connection for new client 859b3993-e1a0-4abd-85d4-455acfbc6fc7 (at 192.168.206.56@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 2873.205094] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2873.210237] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 2873.211488] Lustre: Skipped 2 previous similar messages [ 2873.260406] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:45 to 0x2c0000401:545) [ 2873.261916] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:77 to 0x280000401:577) [ 2893.795610] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2893.803268] LustreError: Skipped 1 previous similar message [ 2893.805301] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2893.819599] Lustre: Skipped 1 previous similar message [ 2893.829097] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2893.835754] Lustre: Skipped 6 previous similar messages [ 2898.341139] Lustre: server umount lustre-OST0000 complete [ 2908.629653] Lustre: server umount lustre-OST0001 complete [ 2918.353568] Lustre: server umount lustre-MDT0000 complete [ 2920.420596] LustreError: 96011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2920.434313] LustreError: 96011:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 40 previous similar messages [ 2923.939879] LustreError: 96018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787830723 with bad export cookie 4812017307071085021 [ 2923.960768] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2923.967930] LustreError: 96018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2925.032300] Lustre: server umount lustre-MDT0001 complete [ 2936.728869] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 07:38:54 (1787830734) [ 2946.428385] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2946.436577] Lustre: *** cfs_fail_loc=197, val=0*** [ 2962.080584] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 2967.201171] LustreError: 103524:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.156@tcp: failed processing log, type 4: rc = -110 [ 2972.363487] Lustre: *** cfs_fail_loc=197, val=0*** [ 2972.365463] Lustre: Skipped 65 previous similar messages [ 2992.930487] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2992.946611] Lustre: Skipped 3 previous similar messages [ 2998.565129] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3005.691538] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3008.731791] Lustre: server umount lustre-OST0000 complete [ 3019.647097] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 07:40:17 (1787830817) [ 3041.745907] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3045.822894] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3056.409716] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3060.870302] LustreError: 105175:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3060.882927] LustreError: 105175:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3060.911563] LustreError: 105175:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 19, retries 0, failed: rc = -5 [ 3061.264816] Lustre: server umount lustre-MDT0000 complete [ 3093.107203] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 3104.354209] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 3140.096845] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3140.297904] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3140.351800] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3140.434717] Lustre: lustre-MDT0000: new disk, initializing [ 3140.530917] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3144.906507] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3155.176509] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3155.317423] Lustre: 109354:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3155.636968] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3159.918096] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3169.378788] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3174.758312] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3182.799731] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3183.170773] Lustre: 111319:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3188.528395] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3190.492685] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3200.492854] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3204.375156] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3204.550305] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3208.418752] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3208.666767] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3214.306687] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3214.309340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3214.330417] LustreError: Skipped 5 previous similar messages [ 3214.354794] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3214.366807] Lustre: Skipped 6 previous similar messages [ 3214.402293] Lustre: Skipped 10 previous similar messages [ 3217.132309] Lustre: server umount lustre-OST0000 complete [ 3236.320168] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3236.532506] Lustre: server umount lustre-MDT0000 complete [ 3239.913815] LustreError: 109364:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3239.954857] LustreError: 109364:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 3240.047718] LustreError: 109344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831039 with bad export cookie 4812017307071095598 [ 3240.055452] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3240.062502] LustreError: 109344:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3240.369713] Lustre: server umount lustre-MDT0001 complete [ 3249.179612] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 07:44:06 (1787831046) [ 3260.030987] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 3263.059921] Key type lgssc unregistered [ 3263.399987] LNet: 113821:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3263.412927] LNetError: 113821:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3263.442290] LNet: Removed LNI 192.168.206.156@tcp [ 3264.626101] Key type .llcrypt unregistered [ 3264.628421] Key type ._llcrypt unregistered [ 3283.671939] Key type ._llcrypt registered [ 3283.673922] Key type .llcrypt registered [ 3283.810300] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 3299.364796] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 3300.428100] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3300.548494] alg: No test for adler32 (adler32-zlib) [ 3301.679168] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 3302.011814] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 3303.747834] Key type lgssc registered [ 3305.208539] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3346.208123] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3346.247696] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3347.497417] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3347.526444] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3347.628329] Lustre: lustre-MDT0000: new disk, initializing [ 3347.697307] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3347.726345] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3351.765654] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3362.026253] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3362.193835] Lustre: lustre-OST0000: new disk, initializing [ 3362.198626] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3362.204098] Lustre: Skipped 1 previous similar message [ 3362.209451] Lustre: 117997:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3362.308456] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3364.001070] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3364.013898] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3364.060797] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3368.362805] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3378.662705] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3378.801474] Lustre: lustre-OST0001: new disk, initializing [ 3378.804665] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3378.809469] Lustre: 118859:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3378.874965] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3385.033915] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3388.548826] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3388.555693] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3388.593502] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3398.626648] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3398.663820] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3398.674498] Lustre: Skipped 1 previous similar message [ 3399.961712] Lustre: server umount lustre-MDT0000 complete [ 3406.098305] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3410.981130] LustreError: 117181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831210 with bad export cookie 16727850178434188317 [ 3410.988232] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3410.991079] LustreError: 117181:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3411.069224] Lustre: server umount lustre-OST0000 complete [ 3414.925140] Lustre: server umount lustre-OST0001 complete [ 3422.927481] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 07:47:00 (1787831220) [ 3441.789938] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 07:47:19 (1787831239) [ 3449.981845] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 3474.119565] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 07:47:51 (1787831271) [ 3508.532496] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 3517.844635] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 3553.584437] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3553.809706] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3553.843742] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3553.949083] Lustre: lustre-MDT0000: new disk, initializing [ 3554.024363] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3554.042983] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3558.600704] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3570.090644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3570.219928] Lustre: 126612:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3570.272327] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3570.276142] Lustre: Skipped 1 previous similar message [ 3570.345923] Lustre: lustre-MDT0001: new disk, initializing [ 3570.441348] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3570.478861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3570.496149] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3575.059136] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3584.738031] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3591.038177] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3597.777398] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3597.999105] Lustre: lustre-OST0000: new disk, initializing [ 3598.003161] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3598.011964] Lustre: 128581:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3598.092354] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3603.955295] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3604.006966] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3604.017512] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3604.153718] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3613.056841] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3617.736678] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3617.911866] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3622.929736] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3623.210444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3630.050082] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3630.058437] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3630.069646] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3630.072742] Lustre: Skipped 2 previous similar messages [ 3630.083411] Lustre: Skipped 1 previous similar message [ 3631.364495] Lustre: server umount lustre-OST0000 complete [ 3635.170989] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3635.181335] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3635.185390] Lustre: Skipped 1 previous similar message [ 3635.197802] Lustre: Skipped 1 previous similar message [ 3640.289967] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3641.037206] Lustre: server umount lustre-MDT0000 complete [ 3644.060444] LustreError: 126604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831443 with bad export cookie 16727850178434189542 [ 3644.068124] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3644.073455] LustreError: 126604:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3644.347291] Lustre: server umount lustre-MDT0001 complete [ 3651.401874] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 07:50:49 (1787831449) [ 3656.481947] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3656.892528] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3660.136216] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3662.317296] LustreError: 130992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3662.338599] LustreError: 130992:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3666.401889] LustreError: 130991:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3667.468525] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3671.488445] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3678.584264] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3684.229930] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3690.985202] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3691.436923] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3691.441506] Lustre: Skipped 1 previous similar message [ 3697.561312] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3705.208849] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3709.451924] Lustre: *** cfs_fail_loc=159, val=0*** [ 3724.896473] Lustre: lustre-MDT0000: Client a1450dcb-51a7-46dc-b5b1-c694e14be152 (at 192.168.206.56@tcp) reconnecting [ 3726.427274] Lustre: *** cfs_fail_loc=159, val=0*** [ 3726.431740] Lustre: Skipped 5 previous similar messages [ 3742.814692] Lustre: lustre-MDT0000: Client a1450dcb-51a7-46dc-b5b1-c694e14be152 (at 192.168.206.56@tcp) reconnecting [ 3763.175737] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3763.181650] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3763.184729] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3763.184737] Lustre: Skipped 1 previous similar message [ 3763.191771] Lustre: Skipped 1 previous similar message [ 3763.214313] LustreError: Skipped 1 previous similar message [ 3765.515809] Lustre: server umount lustre-OST0000 complete [ 3770.337905] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3770.344673] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3773.413138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3773.418963] Lustre: Skipped 3 previous similar messages [ 3775.939379] Lustre: server umount lustre-MDT0000 complete [ 3778.533516] LustreError: 130992:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3778.547551] LustreError: 130992:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 3779.754456] LustreError: 133714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831579 with bad export cookie 16727850178434190417 [ 3779.758061] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3779.765068] LustreError: 133714:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3780.250663] Lustre: server umount lustre-MDT0001 complete [ 3791.325739] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 3793.896486] Key type lgssc unregistered [ 3794.160948] LNet: 134773:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3794.175850] LNetError: 134773:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3794.216924] LNet: Removed LNI 192.168.206.156@tcp [ 3795.160123] Key type .llcrypt unregistered [ 3795.162717] Key type ._llcrypt unregistered [ 3819.756716] Key type ._llcrypt registered [ 3819.759473] Key type .llcrypt registered [ 3819.853414] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 3820.639931] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3820.683957] alg: No test for adler32 (adler32-zlib) [ 3821.701641] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 3821.937829] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 3823.657452] Key type lgssc registered [ 3824.753247] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3833.438405] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3833.451143] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3834.921630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3839.271299] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3845.091216] LustreError: 136188:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3845.106552] LustreError: 136188:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 3847.073256] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3847.438741] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3851.076561] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3856.984204] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3861.203967] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3867.023408] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3867.370959] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3873.034364] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 3879.682927] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 3880.445788] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3888.950666] Lustre: *** cfs_fail_loc=159, val=0*** [ 3905.609227] Lustre: lustre-MDT0000: Client 540f9c69-84e4-4318-b731-ff525644cfe4 (at 192.168.206.56@tcp) reconnecting [ 3907.603410] Lustre: *** cfs_fail_loc=159, val=0*** [ 3923.070337] Lustre: lustre-MDT0001: Client 540f9c69-84e4-4318-b731-ff525644cfe4 (at 192.168.206.56@tcp) reconnecting [ 3924.756314] Lustre: *** cfs_fail_loc=159, val=0*** [ 3924.762298] Lustre: Skipped 3 previous similar messages [ 3939.931079] Lustre: lustre-MDT0001: Client 540f9c69-84e4-4318-b731-ff525644cfe4 (at 192.168.206.56@tcp) reconnecting [ 3944.099118] Lustre: 139221:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3946.858532] Lustre: *** cfs_fail_loc=159, val=0*** [ 3946.861991] Lustre: Skipped 4 previous similar messages [ 3963.484196] Lustre: lustre-MDT0000: Client 23543c07-514b-4fd3-bd0d-3ad18d6654fa (at 192.168.206.56@tcp) reconnecting [ 3965.233865] Lustre: *** cfs_fail_loc=159, val=0*** [ 3965.237092] Lustre: Skipped 14 previous similar messages [ 3980.900406] Lustre: lustre-MDT0000: Client 23543c07-514b-4fd3-bd0d-3ad18d6654fa (at 192.168.206.56@tcp) reconnecting [ 3982.508674] Lustre: 139511:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 3999.722696] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3999.731669] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3999.743430] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4002.787354] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4002.793959] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4002.796197] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4002.832235] Lustre: Skipped 1 previous similar message [ 4004.407694] Lustre: server umount lustre-OST0000 complete [ 4011.491654] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4011.499457] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4011.534950] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4014.429704] Lustre: server umount lustre-MDT0000 complete [ 4018.146220] LustreError: 138943:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4018.177439] LustreError: 138943:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4018.317648] LustreError: 136171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831817 with bad export cookie 14731301776193325793 [ 4018.323712] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4018.327551] LustreError: 136171:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4018.596162] Lustre: server umount lustre-MDT0001 complete [ 4028.404832] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4031.249109] Key type lgssc unregistered [ 4031.575887] LNet: 140592:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4031.598447] LNetError: 140592:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4032.616429] LNet: Removed LNI 192.168.206.156@tcp [ 4033.631737] Key type .llcrypt unregistered [ 4033.633984] Key type ._llcrypt unregistered [ 4062.516297] Key type ._llcrypt registered [ 4062.520281] Key type .llcrypt registered [ 4062.713526] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4063.542334] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4063.592184] alg: No test for adler32 (adler32-zlib) [ 4064.646389] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4064.920206] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4066.632438] Key type lgssc registered [ 4068.030366] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4075.352524] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4075.375903] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4076.836123] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4080.939247] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4087.267996] LustreError: 142011:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4087.283584] LustreError: 142011:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4089.139906] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4089.425357] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4093.210232] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4100.227993] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4105.944723] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4112.588326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4112.885791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4113.916920] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4119.789447] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4122.114611] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4126.551260] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4132.863761] Lustre: 144589:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4135.830194] Lustre: 144638:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4137.170375] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4137.804757] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4140.854734] Lustre: 144784:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4141.735723] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4142.684875] Lustre: 144884:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4147.688218] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4147.693325] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4147.695236] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4147.718034] Lustre: Skipped 1 previous similar message [ 4152.213386] Lustre: server umount lustre-OST0000 complete [ 4155.873190] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4155.885557] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4155.896491] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4155.899199] Lustre: Skipped 1 previous similar message [ 4157.929300] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4157.934125] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4157.944868] Lustre: Skipped 1 previous similar message [ 4161.873222] Lustre: server umount lustre-MDT0000 complete [ 4163.042447] LustreError: 142738:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4163.060413] LustreError: 142738:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4165.262258] LustreError: 141990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787831964 with bad export cookie 400935760995962206 [ 4165.281845] LustreError: 141990:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4165.293490] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4165.599063] Lustre: server umount lustre-MDT0001 complete [ 4175.242553] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4177.998850] Key type lgssc unregistered [ 4178.302772] LNet: 145944:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4178.326394] LNetError: 145944:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4178.352741] LNet: Removed LNI 192.168.206.156@tcp [ 4179.315427] Key type .llcrypt unregistered [ 4179.319538] Key type ._llcrypt unregistered [ 4199.247699] Key type ._llcrypt registered [ 4199.250937] Key type .llcrypt registered [ 4199.353316] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4200.264799] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4200.284983] alg: No test for adler32 (adler32-zlib) [ 4201.400486] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4201.646763] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4203.352213] Key type lgssc registered [ 4204.403564] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4212.000696] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4212.017821] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4213.515050] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4217.855518] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4223.977975] LustreError: 147338:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4224.000720] LustreError: 147338:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4225.721399] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4226.114517] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4230.669387] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4237.909269] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4242.794795] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4249.218819] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4249.597287] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4250.612616] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4254.871345] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4258.812359] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4261.652993] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4267.057844] Lustre: *** cfs_fail_loc=159, val=0*** [ 4267.064243] Lustre: Skipped 3 previous similar messages [ 4283.469233] Lustre: lustre-MDT0000: Client 09c85728-d638-45b5-9114-50668ee3d064 (at 192.168.206.56@tcp) reconnecting [ 4301.793409] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4301.799180] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4301.815606] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4304.867532] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4304.871057] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4304.871200] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4304.880517] Lustre: Skipped 1 previous similar message [ 4306.284381] Lustre: server umount lustre-OST0000 complete [ 4313.058553] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4313.059899] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4313.084770] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4313.098166] Lustre: Skipped 1 previous similar message [ 4315.105476] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4315.112585] Lustre: Skipped 1 previous similar message [ 4317.037330] Lustre: server umount lustre-MDT0000 complete [ 4320.231159] LustreError: 148349:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4320.245816] LustreError: 148349:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4320.834784] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832120 with bad export cookie 615028404060415616 [ 4320.844425] LustreError: 147317:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4320.845189] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4321.243264] Lustre: server umount lustre-MDT0001 complete [ 4330.863803] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4333.306492] Key type lgssc unregistered [ 4333.631518] LNet: 151015:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4333.637701] LNetError: 151015:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4333.668196] LNet: Removed LNI 192.168.206.156@tcp [ 4334.364233] Key type .llcrypt unregistered [ 4334.366059] Key type ._llcrypt unregistered [ 4357.376328] Key type ._llcrypt registered [ 4357.378270] Key type .llcrypt registered [ 4357.495652] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4358.963165] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4359.006106] alg: No test for adler32 (adler32-zlib) [ 4360.036378] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4360.320687] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4362.008463] Key type lgssc registered [ 4363.260517] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4370.418339] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4370.442156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4371.961418] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4375.939775] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4382.182691] LustreError: 152431:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4382.202825] LustreError: 152431:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4384.015440] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4384.338085] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4388.347851] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4394.893098] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4400.529780] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4406.974322] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4407.382337] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4408.430954] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4413.205047] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4416.625077] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4421.189858] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4439.066447] Lustre: lustre-OST0000: evicting 83faef8b-80b4-4d9b-87a6-5861aaa13d13 (at 192.168.206.56@tcp) by administrative request [ 4439.077175] Lustre: Skipped 1 previous similar message [ 4468.204593] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4468.212025] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4468.235849] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4468.242585] Lustre: Skipped 1 previous similar message [ 4473.316181] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4473.797945] Lustre: server umount lustre-OST0000 complete [ 4478.434438] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4478.437067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4478.454044] Lustre: Skipped 2 previous similar messages [ 4478.464411] Lustre: Skipped 2 previous similar messages [ 4483.553113] LustreError: 152432:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4483.566089] LustreError: 152432:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4483.584909] Lustre: server umount lustre-MDT0000 complete [ 4486.921148] LustreError: 153173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832286 with bad export cookie 12270967345126198659 [ 4486.926202] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4486.929164] LustreError: 153173:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4487.200943] Lustre: server umount lustre-MDT0001 complete [ 4495.379506] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4497.771924] Key type lgssc unregistered [ 4498.067272] LNet: 156504:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4498.081627] LNetError: 156504:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4498.100802] LNet: Removed LNI 192.168.206.156@tcp [ 4498.905450] Key type .llcrypt unregistered [ 4498.907790] Key type ._llcrypt unregistered [ 4577.200915] Key type ._llcrypt registered [ 4577.204751] Key type .llcrypt registered [ 4577.365106] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4579.651274] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4579.734059] alg: No test for adler32 (adler32-zlib) [ 4580.841845] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4581.092236] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4582.769299] Key type lgssc registered [ 4584.148346] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4593.008328] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4593.026481] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4594.456640] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4599.190766] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4604.900627] LustreError: 158998:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4604.919994] LustreError: 158998:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4608.216035] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4608.741658] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4613.385382] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4621.152570] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4627.958865] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4634.182196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4634.567835] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4639.735914] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 4640.085267] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4648.444154] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4654.065172] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 4664.290042] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4664.300598] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4664.317340] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4664.318725] Lustre: Skipped 1 previous similar message [ 4664.356206] Lustre: Skipped 1 previous similar message [ 4668.172659] Lustre: server umount lustre-OST0000 complete [ 4674.530809] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4674.534752] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4674.548084] Lustre: Skipped 1 previous similar message [ 4677.641702] Lustre: server umount lustre-MDT0000 complete [ 4679.649599] LustreError: 161495:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4679.667916] LustreError: 161495:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 4681.724813] LustreError: 158977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832481 with bad export cookie 2082750256101388154 [ 4681.746324] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4681.751359] LustreError: 158977:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4682.212968] Lustre: server umount lustre-MDT0001 complete [ 4693.326453] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4695.987515] Key type lgssc unregistered [ 4696.241284] LNet: 162564:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4696.247976] LNetError: 162564:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4696.260818] LNet: Removed LNI 192.168.206.156@tcp [ 4697.028089] Key type .llcrypt unregistered [ 4697.029894] Key type ._llcrypt unregistered [ 4719.141783] Key type ._llcrypt registered [ 4719.143564] Key type .llcrypt registered [ 4719.253899] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4720.325856] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4720.402530] alg: No test for adler32 (adler32-zlib) [ 4721.409322] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4721.685967] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4723.385702] Key type lgssc registered [ 4724.604456] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4740.488834] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 08:08:58 (1787832538) [ 4757.576852] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 4769.303584] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4805.274046] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4805.297816] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4806.549566] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 4806.588752] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 4806.684222] Lustre: lustre-MDT0000: new disk, initializing [ 4806.768118] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4806.799984] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 4811.408810] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4820.769031] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4820.897930] Lustre: 166826:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 4820.949755] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 4820.957623] Lustre: Skipped 1 previous similar message [ 4821.019373] Lustre: lustre-MDT0001: new disk, initializing [ 4821.083683] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4821.106861] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 4821.112115] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 4825.284572] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4836.809982] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4843.636354] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4849.618812] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4849.896981] Lustre: lustre-OST0000: new disk, initializing [ 4849.901092] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 4849.912253] Lustre: 168798:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 4849.976686] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4855.453466] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4858.910205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 4858.923723] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 4859.045076] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 4864.485722] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4874.212759] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4874.223484] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4874.244595] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4874.724951] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4874.726389] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4874.747216] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4879.484564] Lustre: server umount lustre-OST0000 complete [ 4884.970644] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4884.990153] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4889.349718] Lustre: server umount lustre-MDT0000 complete [ 4893.065907] LustreError: 166819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787832692 with bad export cookie 16531679817270870472 [ 4893.074932] LustreError: 166819:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 4893.075559] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4893.448417] Lustre: server umount lustre-MDT0001 complete [ 4903.012001] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 4905.253195] Key type lgssc unregistered [ 4905.603772] LNet: 170703:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4905.614328] LNetError: 170703:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4905.632232] LNet: Removed LNI 192.168.206.156@tcp [ 4906.487592] Key type .llcrypt unregistered [ 4906.491630] Key type ._llcrypt unregistered [ 4932.548298] Key type ._llcrypt registered [ 4932.550602] Key type .llcrypt registered [ 4932.664427] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 4933.852091] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4933.925332] alg: No test for adler32 (adler32-zlib) [ 4935.026777] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 4935.214537] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 4936.848151] Key type lgssc registered [ 4938.053171] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4945.753672] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4945.792897] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4947.227844] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4950.900578] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4956.644860] LustreError: 172118:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4956.660807] LustreError: 172118:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 4958.477829] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4958.877200] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4962.582985] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4969.146864] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4974.061381] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4980.364888] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4980.621768] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4986.317656] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 4991.944349] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 4994.465416] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5000.205223] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5000.236661] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5000.259424] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5000.281284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5000.298839] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5001.403659] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5001.435056] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5001.447090] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5001.456349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5001.476825] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5002.623913] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5002.638980] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5002.646709] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5002.649286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5002.656490] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5003.630716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5004.445690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.163286] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5005.167763] Lustre: Skipped 2 previous similar messages [ 5005.176916] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5005.189531] Lustre: Skipped 2 previous similar messages [ 5005.194154] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5005.199606] Lustre: Skipped 2 previous similar messages [ 5005.204117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5005.221060] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5005.228923] Lustre: Skipped 2 previous similar messages [ 5006.062942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5006.922066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5007.774122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5008.568812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.847647] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5009.858425] Lustre: Skipped 4 previous similar messages [ 5009.875940] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5009.884657] Lustre: Skipped 4 previous similar messages [ 5009.889068] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5009.893963] Lustre: Skipped 4 previous similar messages [ 5009.898981] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5009.908509] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5009.923511] Lustre: Skipped 4 previous similar messages [ 5010.845372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5011.893757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5012.725067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5013.819276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5014.674477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5015.615947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5016.800450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.070325] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5018.081745] Lustre: Skipped 7 previous similar messages [ 5018.109164] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5018.117345] Lustre: Skipped 7 previous similar messages [ 5018.120620] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5018.132631] Lustre: Skipped 7 previous similar messages [ 5018.137649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5018.154908] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5018.170121] Lustre: Skipped 7 previous similar messages [ 5019.316713] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5019.330920] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5019.345445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5020.557824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5021.656450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5022.575314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5023.850131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5024.965384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5026.150540] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5027.123426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5028.133365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5029.305336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5030.527950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5031.899990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5033.211220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5034.054852] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.254209] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5035.261589] Lustre: Skipped 14 previous similar messages [ 5035.283838] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5035.292576] Lustre: Skipped 14 previous similar messages [ 5035.296102] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5035.303765] Lustre: Skipped 14 previous similar messages [ 5035.306482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5035.333917] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5035.341375] Lustre: Skipped 14 previous similar messages [ 5036.813218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5038.281860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5039.528555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5040.642768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5041.461760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5042.269275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5043.164335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5044.138826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5045.122215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5046.092524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.077978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5047.998860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5048.892566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5049.740557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5050.632712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5051.515495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5052.400813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5053.197661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.083259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5054.755422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5055.548956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.255732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5056.958758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5057.694579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5058.491225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5059.327068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.118167] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5060.942739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5061.824927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5062.634040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5063.609694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5064.403089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.200968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5065.905124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5066.650521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.467104] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5067.476929] Lustre: Skipped 35 previous similar messages [ 5067.499971] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5067.518990] Lustre: Skipped 35 previous similar messages [ 5067.524757] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5067.539038] Lustre: Skipped 35 previous similar messages [ 5067.542815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5067.564672] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5067.570488] Lustre: Skipped 35 previous similar messages [ 5068.560777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5069.397102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.235369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5070.897513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5071.742539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5072.567836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5073.501758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5074.504127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5075.333438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.249663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5076.996811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5077.724268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5078.560273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.175095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5079.943983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5080.582390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5081.547178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5082.387914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5083.135713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.094632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5084.891594] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5085.552330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5086.815601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5087.929926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5089.247177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5090.217043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.035370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5091.642978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5092.440518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5093.364766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5094.296208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5095.023894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5096.065584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5097.076070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.027698] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5098.031949] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5098.041492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5098.915625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5099.906684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5100.660547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5101.561735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5102.459413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5103.332338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.079870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5104.950613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5105.741178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5106.484888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5107.177394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.057844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5108.910758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5109.864196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5110.666661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5111.419678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5112.228777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5113.189105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5114.043607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5115.080208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5116.092604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5117.064865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5118.170548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5119.112795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.045766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5120.949566] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5121.932745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5122.727493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5123.533754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5124.455271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5125.209143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.141127] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5126.911085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5127.716224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5128.595676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5129.379577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5130.189026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.054151] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5131.959512] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5131.972525] Lustre: Skipped 73 previous similar messages [ 5131.987062] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5132.002657] Lustre: Skipped 73 previous similar messages [ 5132.010887] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5132.013579] Lustre: Skipped 73 previous similar messages [ 5132.018239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5132.027218] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5132.031657] Lustre: Skipped 73 previous similar messages [ 5132.762477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5133.564106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5134.279968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.095598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5135.802968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5136.580106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5137.334063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5138.169152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5139.241051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5140.416185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5141.401197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5142.323858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.147382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5143.929173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5144.620825] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5145.492658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5146.384747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5147.392618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5148.173567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.023865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5149.859126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5150.695036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5151.548650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5152.375610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5153.283513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5154.259794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5155.218694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.061530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5156.792847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5157.696637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5158.427921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5159.282370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5160.164901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5161.130342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.040712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5162.978418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5163.833557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5164.677451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5165.620368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5166.462318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5167.205672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5168.158754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.060934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5169.877479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5170.642308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5171.623460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5172.519257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5173.479314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5174.213817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.092067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5175.884586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5176.624561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5177.309940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.046561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5178.817450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5179.669396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5180.471510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5181.250460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5182.081192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.023530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5183.834045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5184.835667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5185.587822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5186.635632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5187.445264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5188.398530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5189.190585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5190.233623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.079295] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5191.901768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5192.650526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5193.540649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5194.397798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.150651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5195.945512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5196.883042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5197.780204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5198.785122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5199.921529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5200.979455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5201.939712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5202.957873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5203.875234] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.016043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5205.889736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5206.848053] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5207.836794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5208.731584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5209.521595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5210.567047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5211.672276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5212.662679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5213.710388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5214.743349] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5215.574287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5216.367705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.309867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5217.967465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5218.827899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5219.507228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5220.341144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.060585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5221.826650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5222.541864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5223.438951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5224.228199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.138226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5225.991860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5226.889413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5227.841318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5228.608730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5229.484052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5230.321975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5231.238119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5232.270803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5233.076525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5234.118427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.014075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5235.901474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5236.827267] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5237.779679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5238.675568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5239.635551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5240.524580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5241.409177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5242.480982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5243.381492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5244.298712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5245.207397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5246.072336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.010252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5247.938640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5248.986300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5249.892698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5250.702292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5251.622134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5252.502619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5253.369701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5254.196273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.133528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5255.848819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5256.773187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5257.658700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5258.583162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5259.522804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.375792] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5260.382765] Lustre: Skipped 145 previous similar messages [ 5260.394882] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5260.401994] Lustre: Skipped 145 previous similar messages [ 5260.405674] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5260.415727] Lustre: Skipped 145 previous similar messages [ 5260.419619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5260.430380] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5260.437322] Lustre: Skipped 145 previous similar messages [ 5261.275376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5262.251924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.043519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5263.810760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5264.497125] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.248229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5265.920913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5266.640682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5267.297529] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.005772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5268.798865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5269.551401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5270.605513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5271.495626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5272.493688] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5272.504036] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5272.520877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5273.393479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.317755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5274.969026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5275.744177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5276.543833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5277.686111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5278.991058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5279.814559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5280.625897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5281.433818] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.117301] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5282.129274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5282.129769] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5282.805038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5283.660707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5284.497832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5285.286863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5286.127987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.028975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5287.856532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5288.756471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5289.750511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5290.902078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5291.698920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5292.445278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5293.177096] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.139555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5294.965879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5295.960058] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5296.631681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5297.448858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5298.373188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5299.421894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5300.355074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.098326] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5301.847057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5303.173210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5304.149963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5305.393036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5306.352995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5307.587855] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5307.603989] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5307.628148] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5308.865068] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5310.207707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.048051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5311.976682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5312.714232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5313.697490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5314.530419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5315.310328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5316.717643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5317.877581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5318.690530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5319.636620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.252826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5320.977556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5321.712211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5322.618837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5323.735281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5324.659135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5325.407172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.015649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5326.722249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5327.551869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5328.341224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5329.358238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5330.219418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.052254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5331.984877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5332.901353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5333.718947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5334.574120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5335.609611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5336.450712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5337.517953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5338.460766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5339.514363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5340.640235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5341.603321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5342.360322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5343.316795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5344.412985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5345.427855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5346.339830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5347.148315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.038949] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5348.833630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5349.834901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5350.827348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5351.631851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5352.583671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5353.475656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5354.329072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5355.238967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.132890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5356.997851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5357.832254] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5358.778456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5359.715738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5360.555506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5361.340115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5362.310930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5363.192422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5364.150717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5365.078361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.139698] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5366.874590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5367.781735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5368.696610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5369.488401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5370.348665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5371.384664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5372.255979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5373.241283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.046359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5374.886213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5375.784914] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5376.597517] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5377.498453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5378.381963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.203304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5379.977726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5380.647692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5381.514280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.185607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5382.989037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5383.752270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5384.573651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5385.397557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5386.276475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5387.166390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.004617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5388.840879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5389.579943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5390.431902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.239071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5391.984136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5392.825297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5393.656487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5394.384472] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5395.259560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.050976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5396.862770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5397.719445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5398.541506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5399.353083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.185405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5400.989066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5401.829171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5402.680463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5403.544497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5404.312118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.064108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5405.872520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5406.632790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5407.495033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5408.432241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5409.381052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5410.425584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5411.369648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5412.234324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.087138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5413.922837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5414.694239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5415.584266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5416.397492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.198387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5417.990873] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5418.793283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5419.781602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5420.843184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5421.673893] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5422.633603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5423.439069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5424.251456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.056954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5425.945956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5426.714352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5427.636462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5428.336505] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.154592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5429.930409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5430.873740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5431.575325] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5432.369082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5433.216493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.082520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5434.827803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5435.920335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5436.582668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5437.633862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5438.567868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5439.589078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5440.497613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5441.345054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5442.245431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.102427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5443.950583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5444.825489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5445.786586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5446.585910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5447.387194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.155905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5448.862534] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5449.638233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5450.452245] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5451.268069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5452.136057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5453.132321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5454.131446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.010866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5455.940102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5456.704513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5457.396192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.107952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5458.796717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5459.534585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5460.276806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.033419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5461.908346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5462.639646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5463.402250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5464.155883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.156269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5465.865739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5466.675959] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5467.643696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5468.463129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5469.319975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5470.189263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.028807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5471.975362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5472.655269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5473.462502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5474.298527] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.125806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5475.817237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5476.665573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5477.440436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5478.239844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5479.092832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.003028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5480.830765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5481.882793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5482.792614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5483.627637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5484.441707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5485.420787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5486.351619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5487.198088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5488.116486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5489.183082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5490.188099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.001688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5491.776849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5492.556436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5493.410589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5494.246705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5495.352629] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5496.328140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5497.355370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5498.202419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5499.120478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5500.152188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5501.219310] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5502.360628] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5503.350423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5504.290699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5505.334156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5506.416198] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5507.279633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5508.125131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.049377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5509.997897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5510.612070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5511.537427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5512.484653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5513.537428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5514.541633] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5515.402454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5516.123714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.067230] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5517.075228] Lustre: Skipped 293 previous similar messages [ 5517.098650] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5517.106959] Lustre: Skipped 293 previous similar messages [ 5517.112025] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5517.115238] Lustre: Skipped 293 previous similar messages [ 5517.122177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5517.138374] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5517.143930] Lustre: Skipped 293 previous similar messages [ 5518.010284] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.049924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.971894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5521.119619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.025134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.014978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5524.235668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.457137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.386581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.295255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5528.183579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.064525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.987795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.829506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.821315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.747768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5533.734139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.623619] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.531597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.470653] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5537.460324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5538.358067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.205995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.123228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.025356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.908218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.770721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5543.659768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.517134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.407580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.241504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.064726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.849832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.854668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.879343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5550.745130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.715910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.373359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.298408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.060617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5554.954623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.633163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.415181] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5556.426456] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5556.430213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.112794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.957260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.632674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.503644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.327161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.209062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.915042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.899119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.802948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.720662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.646266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.529287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.539679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.547567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5569.550842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.508550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.444774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.280413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.999704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.834439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.660489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.488952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.287559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.162881] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 5577.172241] LustreError: 172138:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 5577.178174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.861178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.685656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.499846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.289532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.027105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.926903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.684915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.478545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.244754] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.159844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.979384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.107546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.969478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.945719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.766667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.554479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.326784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.123216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5592.908656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.782145] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5594.673842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.571213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.315870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.140315] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5597.911153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.754569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.526574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5600.479741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.195037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.110176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5602.899596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.766364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.480898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.244496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.920464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5606.644575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.417384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5608.218637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.047079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.845659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.811982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5611.617219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5612.404605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.241887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.043064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.972626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.752963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5616.590670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.427816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.422032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.119333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.933473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.720694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.583571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5622.374749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.135572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.888407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.868448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5625.656185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.619259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.493577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.350076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.189894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.152157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.124548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.952070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.928834] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5633.872751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.892962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5635.799455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.725679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.652419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.558035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.461232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.257108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.017509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.874795] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.686124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.491746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.387752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.210259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.952546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.621338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.475448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.286951] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.969687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.725696] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5650.705386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.463311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.210881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5652.897249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5653.720585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.650136] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.590859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.391984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.180390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5658.086270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.203353] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5660.088614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.093195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.243944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.160728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.287966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.376328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.430740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.517314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.597271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.409048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5670.333895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.090129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.024918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.985878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.000676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.901098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.900351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.853945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.794632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.797647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.830467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.528334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.279622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.197598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.029211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.887494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.830565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.708421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.583515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.597073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.307630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.093839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.819379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5690.652392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.426403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.316233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.222248] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.097920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.751149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.582945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.459269] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.382490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.231026] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.054227] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5699.663048] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.484816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.069858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.695617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.309091] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.075437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.745428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.605095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.361847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.177741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.952963] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.822565] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.591497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5709.408904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.214988] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.116261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.041327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.899569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.718123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.498854] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.275428] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.224537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.115477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5717.966453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.804581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.659300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.488977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.324135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.083771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.770130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5723.577074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.340448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.194889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.880738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.803590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.508466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.311824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.122486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.086985] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.792075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.519235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.318376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.249704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.002368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5734.802274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.554946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.284905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.028031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.766623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5738.487570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.367506] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.088616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.987126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.783724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.661035] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5743.394773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.214270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.909303] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.643387] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.316568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.999709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.813607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.675138] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5749.481879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.480373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.582592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.507394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.547149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.557905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.481657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.385155] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.170378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.006683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.909586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5759.707790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.526609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.268962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.096984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.973480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.876308] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.770755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.736324] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.699297] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.525671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.276393] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.147435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5770.020602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.100703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.905465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.692444] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.552510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.399299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.281090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.297879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.414080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.348883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.279822] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.106842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5780.948318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.763823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.545897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.367233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.139405] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.963033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.710494] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5786.565455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.486829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.372608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.127521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.973483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.771383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5791.605856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.388816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.199427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.982020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.978470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.772817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.801277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5797.640530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.693046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.621576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.581602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.491020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.409655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.273588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.274859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.210453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.159489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.951133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.936636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.868262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.692574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.616365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.486550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.433826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.290757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.939719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.690164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.541120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.592554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.482559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.868717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.109495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.139076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.053418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.931518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.770838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.849989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.710408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.357923] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.038775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.680774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5828.409371] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.134929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.849735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.669163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.382401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.104350] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.846900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.604950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.462650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.314831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.248785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.001072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.933915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.687554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.566328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.604235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.397699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.267480] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.223918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.048163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.836274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.763694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.584360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.435078] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.160055] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.014235] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.821278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.736338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.855687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.747355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.723859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.560160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.406120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.350551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.304192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.172064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.969761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.892486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.804279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5861.753244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.800143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.681215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.666676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.590331] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.498620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.537385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.358747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.226247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.236520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.307067] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.438947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.501944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.326312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5875.423066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.327664] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.502067] LustreError: 173938:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 6260.519868] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 6260.530763] LustreError: 172138:0:(osp_precreate.c:655:osp_precreate_send()) Skipped 1 previous similar message [ 6898.659933] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6898.670366] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6898.682753] Lustre: Skipped 416 previous similar messages [ 6898.690648] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6900.199758] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 6900.210333] LustreError: Skipped 4 previous similar messages [ 6900.211799] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6904.219976] Lustre: server umount lustre-OST0000 complete [ 6909.414276] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6909.429227] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6909.434546] Lustre: Skipped 1 previous similar message [ 6913.734419] Lustre: server umount lustre-MDT0000 complete [ 6916.068828] LustreError: 172119:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6916.082919] LustreError: 172119:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 6917.283080] LustreError: 172100:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834716 with bad export cookie 5556710042098230569 [ 6917.292650] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6917.664896] Lustre: server umount lustre-MDT0001 complete [ 6927.469751] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 6929.765650] Key type lgssc unregistered [ 6930.035369] LNet: 227357:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6930.052300] LNetError: 227357:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6930.072369] LNet: Removed LNI 192.168.206.156@tcp [ 6931.033637] Key type .llcrypt unregistered [ 6931.037635] Key type ._llcrypt unregistered [ 6957.815799] Key type ._llcrypt registered [ 6957.822692] Key type .llcrypt registered [ 6957.953196] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 6959.410903] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6959.448123] alg: No test for adler32 (adler32-zlib) [ 6960.513905] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 6960.727987] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 6962.432230] Key type lgssc registered [ 6963.693792] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6970.972789] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6971.008605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6972.578538] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6976.615372] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 6983.143320] LustreError: 228773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6983.158779] LustreError: 228773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 6985.035604] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6985.482554] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6990.107605] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 6998.149471] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7003.597142] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7011.650642] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7012.266086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7018.442596] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 7019.467408] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7027.718573] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7036.410881] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 7038.350503] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7038.360732] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 7038.366583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7038.379163] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 7038.380370] LustreError: 228793:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 7040.576311] Lustre: lustre-OST0000: Not available for connect from 192.168.206.56@tcp (stopping) [ 7040.998168] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7041.021716] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7045.671199] Lustre: lustre-OST0000: Not available for connect from 192.168.206.56@tcp (stopping) [ 7045.678401] Lustre: Skipped 2 previous similar messages [ 7046.581201] Lustre: server umount lustre-OST0000 complete [ 7058.448592] LustreError: 228793:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 7080.448715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7080.824427] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7086.304822] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7092.995676] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7096.175224] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.186785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7096.188331] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 7096.502545] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 2 sec [ 7101.181526] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c8585493800.ost_server_uuid 50 [ 7102.312747] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c8585493800.ost_server_uuid in FULL state after 0 sec [ 7106.534360] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7106.545333] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7106.550485] Lustre: Skipped 1 previous similar message [ 7106.552365] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7106.564963] Lustre: Skipped 3 previous similar messages [ 7111.549626] Lustre: server umount lustre-OST0000 complete [ 7116.770565] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7116.772183] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7116.790216] Lustre: Skipped 2 previous similar messages [ 7121.025941] Lustre: server umount lustre-MDT0000 complete [ 7121.890218] LustreError: 228773:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7121.915709] LustreError: 228773:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7124.233572] LustreError: 230618:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787834923 with bad export cookie 17824204155802906997 [ 7124.243228] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7124.509765] Lustre: server umount lustre-MDT0001 complete [ 7133.317639] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 7135.405879] Key type lgssc unregistered [ 7135.639810] LNet: 233989:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7135.648470] LNetError: 233989:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7135.659940] LNet: Removed LNI 192.168.206.156@tcp [ 7136.280454] Key type .llcrypt unregistered [ 7136.285211] Key type ._llcrypt unregistered [ 7161.687716] Key type ._llcrypt registered [ 7161.690043] Key type .llcrypt registered [ 7161.780238] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 7162.790096] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7162.821969] alg: No test for adler32 (adler32-zlib) [ 7163.858656] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 7164.052895] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 7165.696163] Key type lgssc registered [ 7166.583186] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7174.343578] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7174.383958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7175.818063] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7180.071926] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7186.405652] LustreError: 235697:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7186.426929] LustreError: 235697:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7188.657964] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7189.091790] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7192.885189] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7195.503466] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7201.734461] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7202.174592] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7209.390900] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7218.672820] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7218.821346] Lustre: lustre-OST0001: new disk, initializing [ 7218.824980] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7218.833275] Lustre: 238005:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7218.925752] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7222.014436] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 7224.362480] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7226.932532] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7226.939198] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7227.039730] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7234.354255] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7243.500981] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7258.083547] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7258.100580] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7258.109462] Lustre: Skipped 3 previous similar messages [ 7258.117922] Lustre: Skipped 3 previous similar messages [ 7263.202879] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7263.219209] Lustre: Skipped 2 previous similar messages [ 7268.323131] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7268.340336] Lustre: Skipped 3 previous similar messages [ 7271.392176] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7271.650081] Lustre: server umount lustre-MDT0000 complete [ 7273.454138] LustreError: 235697:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7273.470955] LustreError: 235697:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 7275.615603] LustreError: 236433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835075 with bad export cookie 6410527845467506651 [ 7275.627846] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7275.631483] LustreError: 236433:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7276.012349] Lustre: server umount lustre-MDT0001 complete [ 7289.563653] Lustre: server umount lustre-OST0000 complete [ 7303.289535] Lustre: server umount lustre-OST0001 complete [ 7325.128399] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 7334.887651] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7335.309726] LustreError: 241113:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7335.395188] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7339.312064] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7340.531569] LustreError: 241114:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7344.611692] LustreError: 241113:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7349.777476] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7350.618356] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7355.967601] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7359.597881] Lustre: 242254:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7368.049271] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7368.369892] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7373.437766] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7374.537400] Lustre: mylustre-OST0000: new connection from mylustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:100 to 0x280000400:129) [ 7375.592802] LustreError: 242613:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7375.626402] LustreError: 242613:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 7379.694937] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49760) [ 7381.391582] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7387.315447] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7394.455865] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7397.781412] Lustre: 244129:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7407.093412] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7407.111691] LustreError: 241114:0:(ldlm_lib.c:1190:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7407.124936] LustreError: 241114:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 7407.206912] Lustre: server umount mylustre-MDT0000 complete [ 7410.528672] LustreError: 244329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835210 with bad export cookie 6410527845467509514 [ 7410.534083] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7410.541502] LustreError: 244329:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7412.197792] Lustre: mylustre-MDT0001-lwp-OST0000: Connection to mylustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7412.200487] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7412.215894] Lustre: Skipped 4 previous similar messages [ 7416.960118] Lustre: server umount mylustre-MDT0001 complete [ 7421.433780] Lustre: server umount mylustre-OST0000 complete [ 7425.280663] Lustre: server umount mylustre-OST0001 complete [ 7444.302870] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 7452.865420] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7453.182215] LustreError: 246434:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7453.198311] LustreError: 246434:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 7 previous similar messages [ 7453.253221] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7453.257826] Lustre: Skipped 1 previous similar message [ 7456.963496] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7464.054810] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7468.396296] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7471.163511] Lustre: 247575:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7476.737472] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7479.023427] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49792) [ 7482.268897] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7488.504341] Lustre: tfs-OST0000: new connection from tfs-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:161) [ 7489.725469] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7496.315102] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7504.266753] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7508.220370] Lustre: 249446:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7515.624547] Lustre: tfs-MDT0000-lwp-OST0000: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7515.625117] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7515.639268] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 7515.650302] Lustre: Skipped 6 previous similar messages [ 7520.743189] LustreError: 247167:0:(ldlm_lib.c:1190:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7520.757047] LustreError: 247167:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 6 previous similar messages [ 7520.843436] Lustre: server umount tfs-MDT0000 complete [ 7524.048992] LustreError: 246409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835323 with bad export cookie 6410527845467511467 [ 7524.055881] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7524.063280] LustreError: 246409:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7524.298542] Lustre: server umount tfs-MDT0001 complete [ 7536.677543] Lustre: server umount tfs-OST0000 complete [ 7550.046822] Lustre: server umount tfs-OST0001 complete [ 7581.152957] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 7589.589323] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7589.983517] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7589.986956] Lustre: Skipped 3 previous similar messages [ 7593.536968] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7600.964798] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7607.282164] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7610.453789] Lustre: 252841:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7619.816754] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7625.469442] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:131 to 0x280000400:193) [ 7627.752475] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7634.942207] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49824) [ 7636.829954] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7642.869852] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7650.110887] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7653.613906] Lustre: 254711:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7660.807964] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 08:57:38 (1787835458) [ 7677.924924] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7677.927806] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7677.945135] Lustre: Skipped 5 previous similar messages [ 7677.954367] Lustre: Skipped 3 previous similar messages [ 7681.819685] Lustre: server umount lustre-MDT0000 complete [ 7683.046720] LustreError: 251699:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7683.059869] LustreError: 251699:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 7685.450359] LustreError: 252843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835485 with bad export cookie 6410527845467513161 [ 7685.461906] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7685.466694] LustreError: 252843:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 7688.163195] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7688.180563] Lustre: Skipped 2 previous similar messages [ 7691.918198] Lustre: server umount lustre-MDT0001 complete [ 7701.124917] Lustre: server umount lustre-OST0000 complete [ 7710.620448] Lustre: server umount lustre-OST0001 complete [ 7716.212515] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 7724.973503] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 7756.081511] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7756.281712] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7756.312285] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7756.418742] Lustre: lustre-MDT0000: new disk, initializing [ 7756.503401] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 188-180 [ 7756.510046] Lustre: Skipped 3 previous similar messages [ 7756.535639] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7760.495978] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7771.052056] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7771.151984] Lustre: 257892:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7771.196192] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7771.200358] Lustre: Skipped 1 previous similar message [ 7771.268263] Lustre: lustre-MDT0001: new disk, initializing [ 7771.425331] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7771.448976] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7775.908942] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7785.735672] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7785.970196] Lustre: lustre-OST0000: new disk, initializing [ 7785.976071] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7785.983567] Lustre: 259474:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7787.860476] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7787.870995] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7788.020954] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7792.487768] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7803.216254] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7803.491891] Lustre: lustre-OST0001: new disk, initializing [ 7803.495157] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7803.502386] Lustre: 260344:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7805.478656] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7805.490277] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7805.626583] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7809.834671] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 7822.810383] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 09:00:20 (1787835620) [ 7838.006823] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 09:00:35 (1787835635) [ 7854.561866] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7854.569340] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7854.589441] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7854.601328] Lustre: Skipped 5 previous similar messages [ 7868.384669] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7868.554817] Lustre: server umount lustre-OST0000 complete [ 7874.018604] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7874.032163] LustreError: Skipped 1 previous similar message [ 7878.792231] Lustre: server umount lustre-MDT0000 complete [ 7882.488579] LustreError: 257883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835682 with bad export cookie 6410527845467514715 [ 7882.490580] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7882.504455] LustreError: 257883:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7887.331665] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7887.351872] Lustre: Skipped 4 previous similar messages [ 7887.358581] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7887.365170] Lustre: Skipped 10 previous similar messages [ 7889.005854] Lustre: server umount lustre-MDT0001 complete [ 7902.074329] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 7972.107355] Lustre: server umount lustre-OST0001 complete [ 7978.129698] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 7996.644866] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 8031.545960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8031.748160] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8031.808202] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8031.933565] Lustre: lustre-MDT0000: new disk, initializing [ 8031.987987] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8031.990588] Lustre: Skipped 3 previous similar messages [ 8032.007664] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8036.630475] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8049.109644] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8049.299925] Lustre: 265201:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8049.398186] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8049.403597] Lustre: Skipped 1 previous similar message [ 8049.530490] Lustre: lustre-MDT0001: new disk, initializing [ 8049.738229] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8049.752393] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8055.077819] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8064.460325] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8070.266873] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8077.299655] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8077.539461] Lustre: 267169:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8078.755475] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8078.858267] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8083.395041] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8091.954464] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8099.297473] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8099.309597] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8099.318594] LustreError: Skipped 1 previous similar message [ 8099.319621] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8099.341368] Lustre: Skipped 1 previous similar message [ 8103.719996] Lustre: server umount lustre-OST0000 complete [ 8113.894387] Lustre: server umount lustre-MDT0000 complete [ 8114.657872] LustreError: 265211:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8114.693821] LustreError: 265211:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 13 previous similar messages [ 8117.883588] LustreError: 265192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787835917 with bad export cookie 6410527845467516185 [ 8117.896540] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8117.901986] LustreError: 265192:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8118.248741] Lustre: server umount lustre-MDT0001 complete [ 8129.322226] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 8132.328546] Key type lgssc unregistered [ 8132.715098] LNet: 268931:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8132.726558] LNetError: 268931:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8132.749856] LNet: Removed LNI 192.168.206.156@tcp [ 8133.648271] Key type .llcrypt unregistered [ 8133.652391] Key type ._llcrypt unregistered [ 8158.822458] Key type ._llcrypt registered [ 8158.824796] Key type .llcrypt registered [ 8158.972544] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 8160.213962] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8160.272582] alg: No test for adler32 (adler32-zlib) [ 8161.354865] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 8161.736928] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 8163.408162] Key type lgssc registered [ 8164.286601] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8172.051621] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8172.059222] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8173.362371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8177.665057] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8182.777769] LustreError: 270396:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8182.813569] LustreError: 270396:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [ 8187.600758] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8187.878182] LustreError: 270395:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8188.260695] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8193.700749] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8202.159144] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8208.954690] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8216.473983] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8216.788028] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8217.867849] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 8223.393273] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8231.556807] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8232.596547] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 8238.563244] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8238.572456] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8238.584268] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8238.594194] Lustre: Skipped 1 previous similar message [ 8238.609400] Lustre: Skipped 1 previous similar message [ 8240.349699] Lustre: server umount lustre-OST0000 complete [ 8243.687363] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8243.696375] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8243.708978] Lustre: Skipped 1 previous similar message [ 8248.803377] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8248.810040] Lustre: Skipped 2 previous similar messages [ 8249.846365] Lustre: server umount lustre-MDT0000 complete [ 8252.921473] LustreError: 272923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836052 with bad export cookie 11326431268757070867 [ 8252.926389] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8252.933148] LustreError: 272923:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8253.252619] Lustre: server umount lustre-MDT0001 complete [ 8266.663470] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 8269.741340] Key type lgssc unregistered [ 8270.113709] LNet: 273982:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8270.131518] LNetError: 273982:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8270.173471] LNet: Removed LNI 192.168.206.156@tcp [ 8271.265552] Key type .llcrypt unregistered [ 8271.269576] Key type ._llcrypt unregistered [ 8287.809659] Key type ._llcrypt registered [ 8287.812232] Key type .llcrypt registered [ 8287.907705] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 8288.758146] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8288.783360] alg: No test for adler32 (adler32-zlib) [ 8289.937573] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 8290.155121] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 8291.872165] Key type lgssc registered [ 8292.730151] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8300.373962] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8300.403868] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8315.936455] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 8321.056361] LustreError: 275180:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.206.156@tcp: failed processing log, type 4: rc = -110 [ 8347.681084] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8354.308311] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8361.610819] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8365.155403] Lustre: server umount lustre-OST0000 complete [ 8383.484908] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 8386.949396] Key type lgssc unregistered [ 8387.171640] LNet: 276814:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8387.178863] LNetError: 276814:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8387.200212] LNet: Removed LNI 192.168.206.156@tcp [ 8388.641114] Key type .llcrypt unregistered [ 8388.645842] Key type ._llcrypt unregistered [ 8431.612761] Key type ._llcrypt registered [ 8431.615426] Key type .llcrypt registered [ 8431.743471] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 8433.196656] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8433.222747] alg: No test for adler32 (adler32-zlib) [ 8434.330608] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 8434.554243] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 8436.248169] Key type lgssc registered [ 8437.591207] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8449.674378] loop0: detected capacity change from 0 to 204800000 [ 8449.831320] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8449.843891] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8450.080237] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8451.217494] loop0: detected capacity change from 0 to 204800000 [ 8451.439867] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8451.468823] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8451.908469] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8452.787513] loop0: detected capacity change from 0 to 204800000 [ 8452.914231] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8452.922233] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8452.993920] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8454.028367] loop0: detected capacity change from 0 to 204800000 [ 8454.201649] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 8455.174036] loop0: detected capacity change from 0 to 204800000 [ 8455.201280] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8458.768494] loop0: detected capacity change from 0 to 204800000 [ 8458.794398] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8462.258255] loop0: detected capacity change from 0 to 204800000 [ 8462.282981] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8465.726651] loop0: detected capacity change from 0 to 204800000 [ 8465.743821] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 8468.582634] loop0: detected capacity change from 0 to 204800000 [ 8468.660380] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8468.716581] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8468.789202] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8468.796466] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 8468.809443] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8468.825827] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8468.845203] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 8468.853166] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8468.870733] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 8468.885492] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 8468.897374] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 8468.908049] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 8468.921494] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 8468.934434] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 8468.943528] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 8468.951126] Lustre: 280225:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 8474.099173] Lustre: server umount MGS complete [ 8474.832467] loop0: detected capacity change from 0 to 204800000 [ 8474.897701] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8475.373354] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8475.430434] Lustre: lustre-MDT0000: Aborting client recovery [ 8476.203165] loop1: detected capacity change from 0 to 204800000 [ 8476.310758] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8476.390919] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8476.402831] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 8476.414592] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8476.435637] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 8476.448422] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8476.462672] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8476.478976] Lustre: 280656:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 8476.510657] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8476.745718] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8476.792795] Lustre: lustre-MDT0001: Aborting client recovery [ 8477.543453] loop2: detected capacity change from 0 to 204800000 [ 8477.627651] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8477.721886] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8477.735632] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 8477.754022] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8477.767523] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8477.785710] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 8477.797885] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8477.807096] Lustre: 280738:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 8477.965836] Lustre: Found index 0 for lustre-OST0000, updating log [ 8478.000392] Lustre: 280756:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8478.069675] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8478.092210] Lustre: lustre-OST0000: Aborting recovery [ 8478.816312] loop3: detected capacity change from 0 to 204800000 [ 8478.982143] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8479.072767] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 8479.083143] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 8479.097675] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 8479.111542] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 8479.124925] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 8479.136864] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 8479.146428] Lustre: 280822:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 8479.166720] Lustre: Found index 1 for lustre-OST0001, updating log [ 8479.191992] Lustre: 280827:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8479.256468] Lustre: lustre-OST0001: Aborting recovery [ 8484.351604] LustreError: 280600:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 8, retries 0, failed: rc = -108 [ 8485.356648] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8485.357467] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 8485.422439] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 8485.423471] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 8487.407538] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 8487.410757] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 8487.893242] LustreError: 280567:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 156us: rc = -2 [ 8488.398438] LustreError: 280567:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 506us: rc = -2 [ 8488.430926] LustreError: 280567:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 23 previous similar messages [ 8502.240685] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8502.247903] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8502.259885] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8502.754064] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8502.769375] Lustre: Skipped 2 previous similar messages [ 8502.773762] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8502.777799] Lustre: Skipped 2 previous similar messages [ 8507.110499] Lustre: server umount lustre-MDT0000 complete [ 8507.881924] LustreError: 280570:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8507.893167] LustreError: 280570:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [ 8507.915625] LustreError: 280550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836307 with bad export cookie 15530101547170893406 [ 8507.917186] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8507.929733] LustreError: 280550:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8512.996261] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8513.000697] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8513.008824] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8513.012829] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8513.039121] Lustre: Skipped 1 previous similar message [ 8518.112994] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8518.113957] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8518.134160] LustreError: 280565:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8518.147117] Lustre: Skipped 2 previous similar messages [ 8522.208119] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8522.399069] Lustre: server umount lustre-MDT0001 complete [ 8529.517630] Lustre: server umount lustre-OST0000 complete [ 8536.659311] Lustre: server umount lustre-OST0001 complete [ 8546.179814] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 09:12:23 (1787836343) [ 8574.182614] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 8584.489268] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 8629.854330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8630.156984] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8630.208093] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8630.348776] Lustre: lustre-MDT0000: new disk, initializing [ 8630.436661] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8630.443356] Lustre: Skipped 1 previous similar message [ 8630.454367] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8635.217597] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8647.122767] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8647.258969] Lustre: 284899:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8647.366921] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8647.373505] Lustre: Skipped 1 previous similar message [ 8647.432518] Lustre: lustre-MDT0001: new disk, initializing [ 8647.536511] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8647.616916] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8647.646921] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8652.877208] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8663.261362] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8669.436823] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8677.197297] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8677.520563] Lustre: lustre-OST0000: new disk, initializing [ 8677.526388] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8677.539291] Lustre: 286866:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8677.675154] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8684.157095] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8684.170493] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8684.266015] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8685.239854] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8695.752401] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8701.673358] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 8711.227612] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 8719.066193] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 8726.011607] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 8771.559036] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8771.559131] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8771.565705] LustreError: Skipped 1 previous similar message [ 8771.582413] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8771.582423] Lustre: Skipped 2 previous similar messages [ 8773.534705] Lustre: server umount lustre-OST0000 complete [ 8780.770973] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8780.777619] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8780.788013] Lustre: Skipped 1 previous similar message [ 8780.791713] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8780.796480] Lustre: Skipped 1 previous similar message [ 8783.477157] Lustre: server umount lustre-MDT0000 complete [ 8786.914963] LustreError: 284904:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8786.932508] LustreError: 284904:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 8787.059941] LustreError: 284892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836586 with bad export cookie 15530101547170896108 [ 8787.068527] LustreError: 284892:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 8787.069324] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8787.392647] Lustre: server umount lustre-MDT0001 complete [ 8795.620248] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8800.073690] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8805.062711] Lustre: server umount MGS complete [ 8817.509518] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8818.035395] LustreError: 290733:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8818.060815] LustreError: 290746:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 8818.110764] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8822.609954] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8828.386463] LustreError: 290734:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8828.421604] LustreError: 290734:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 8831.478574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8836.089568] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8843.082292] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8848.686489] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8855.666963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8856.009855] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8856.015888] Lustre: Skipped 1 previous similar message [ 8862.082900] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 8869.203811] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8899.043784] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8899.052511] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8899.063044] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8899.072666] Lustre: Skipped 2 previous similar messages [ 8899.084462] Lustre: Skipped 3 previous similar messages [ 8912.352143] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8912.632874] Lustre: server umount lustre-OST0000 complete [ 8919.010534] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8919.031665] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8930.272452] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8930.407709] Lustre: server umount lustre-MDT0000 complete [ 8933.754894] LustreError: 290715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787836733 with bad export cookie 15530101547170897963 [ 8933.758365] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8933.765296] LustreError: 290715:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8934.254534] Lustre: server umount lustre-MDT0001 complete [ 8944.972554] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 8947.783288] Key type lgssc unregistered [ 8948.048611] LNet: 294684:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8948.060057] LNetError: 294684:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8948.077348] LNet: Removed LNI 192.168.206.156@tcp [ 8948.846294] Key type .llcrypt unregistered [ 8948.855267] Key type ._llcrypt unregistered [ 8986.465882] Key type ._llcrypt registered [ 8986.469136] Key type .llcrypt registered [ 8986.616547] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 9001.912488] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 9003.522168] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9003.567676] alg: No test for adler32 (adler32-zlib) [ 9004.607528] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 9004.978790] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 9006.724091] Key type lgssc registered [ 9008.240632] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9048.347606] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9048.380286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9049.636749] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9049.700645] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9049.800740] Lustre: lustre-MDT0000: new disk, initializing [ 9049.942441] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9049.969578] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9054.651559] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9065.247171] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9065.352670] Lustre: 298644:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9065.434228] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9065.438629] Lustre: Skipped 1 previous similar message [ 9065.520127] Lustre: lustre-MDT0001: new disk, initializing [ 9065.626852] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9065.664205] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9065.671457] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9069.810523] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9081.340847] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9088.122616] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9095.704169] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9095.978622] Lustre: lustre-OST0000: new disk, initializing [ 9095.981920] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9096.000774] Lustre: 300616:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9096.126252] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9103.006808] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9103.383273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9103.392738] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9103.551073] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9112.791406] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9121.290246] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9132.931955] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9136.342749] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 9146.319899] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 9195.491084] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9195.501161] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9195.540461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9196.003586] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9196.006845] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9196.058944] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9196.068274] Lustre: Skipped 1 previous similar message [ 9197.586230] Lustre: server umount lustre-OST0000 complete [ 9204.192908] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9204.197950] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9204.205742] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9206.250138] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9206.252229] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9206.264203] Lustre: Skipped 1 previous similar message [ 9208.386477] Lustre: server umount lustre-MDT0000 complete [ 9211.365836] LustreError: 299774:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9211.379894] LustreError: 299774:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [ 9211.581903] LustreError: 298636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837011 with bad export cookie 11077725393518784560 [ 9211.585616] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9211.593831] LustreError: 298636:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9211.817742] Lustre: server umount lustre-MDT0001 complete [ 9221.043583] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9226.512187] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9232.013689] Lustre: server umount MGS complete [ 9244.904934] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9245.628580] LustreError: 304440:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9245.660961] LustreError: 304453:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 9245.804902] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9250.786985] LustreError: 304441:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9252.429733] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9255.909073] LustreError: 304440:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9261.027335] LustreError: 304441:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9262.836273] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9263.117392] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9268.054039] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9276.636743] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9282.614321] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9290.251975] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9290.814919] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9297.857782] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9305.464190] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9331.682075] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 9331.687456] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9331.696129] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9336.776498] Lustre: server umount lustre-OST0000 complete [ 9343.974169] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9343.995629] Lustre: Skipped 2 previous similar messages [ 9343.999638] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9344.014540] Lustre: Skipped 3 previous similar messages [ 9355.232763] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9355.648638] Lustre: server umount lustre-MDT0000 complete [ 9359.334953] LustreError: 304441:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9359.363238] LustreError: 304441:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [ 9359.816752] LustreError: 304420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787837159 with bad export cookie 11077725393518786513 [ 9359.826543] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9359.827146] LustreError: 304420:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 9360.152772] Lustre: server umount lustre-MDT0001 complete [ 9371.170955] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [ 9373.964872] Key type lgssc unregistered [ 9374.336474] LNet: 308390:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9374.354697] LNetError: 308390:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9375.403183] LNet: Removed LNI 192.168.206.156@tcp [ 9376.274291] Key type .llcrypt unregistered [ 9376.277889] Key type ._llcrypt unregistered [ 9396.147445] Key type ._llcrypt registered [ 9396.149904] Key type .llcrypt registered [ 9396.245656] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [ 9409.442545] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 9410.355521] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9410.443164] alg: No test for adler32 (adler32-zlib) [ 9411.504545] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [ 9411.790034] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [ 9413.544177] Key type lgssc registered [ 9414.636535] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9438.043473] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [ 9453.190838] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9453.203328] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9454.365288] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9454.391925] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9454.449723] Lustre: lustre-MDT0000: new disk, initializing [ 9454.517630] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9454.527283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9458.418852] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9468.873939] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9469.229312] Lustre: lustre-OST0000: new disk, initializing [ 9469.236060] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9469.244146] Lustre: Skipped 1 previous similar message [ 9469.254387] Lustre: 312444:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9469.321573] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9474.525799] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [ 9477.200277] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 9477.213537] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 9477.244136] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10316.771418] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10316.798408] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10319.041143] Lustre: server umount lustre-MDT0000 complete [10323.005583] LustreError: 311631:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838122 with bad export cookie 9167569215072939099 [10323.021971] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10323.194725] Lustre: server umount lustre-OST0000 complete [10348.047338] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [10357.717512] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [10397.440823] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10397.688410] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10397.711673] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10397.802684] Lustre: lustre-MDT0000: new disk, initializing [10397.864216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10397.882492] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10402.724739] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10413.748961] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10413.943228] Lustre: 316787:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10414.001590] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10414.004476] Lustre: Skipped 1 previous similar message [10414.128202] Lustre: lustre-MDT0001: new disk, initializing [10414.252844] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10414.293542] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10414.310162] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10418.738769] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10428.502612] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10433.683422] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10441.146359] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10441.706307] Lustre: lustre-OST0000: new disk, initializing [10441.711868] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10441.726850] Lustre: 318755:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10441.831776] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10449.499500] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10449.515678] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10449.653506] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10450.092744] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10459.695442] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10464.338324] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10464.521654] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10468.888487] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [10469.119645] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10475.494567] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10475.496047] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10475.523088] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10477.425334] Lustre: server umount lustre-OST0000 complete [10485.732667] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10485.745055] Lustre: Skipped 1 previous similar message [10485.751191] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10485.764156] Lustre: Skipped 2 previous similar messages [10488.187928] Lustre: server umount lustre-MDT0000 complete [10492.095354] LustreError: 316778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838291 with bad export cookie 9167569215075879911 [10492.105288] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10492.108310] LustreError: 316778:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [10492.494218] Lustre: server umount lustre-MDT0001 complete [10496.369975] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [10498.521488] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 09:44:55 (1787838295) [10504.860281] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10505.303815] LustreError: 321065:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10505.322835] LustreError: 321065:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10505.384499] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10510.824684] LustreError: 321066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10510.961797] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10514.914773] LustreError: 321065:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10520.038846] LustreError: 321066:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10522.223480] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10522.853745] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10528.276343] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10535.970930] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10542.314858] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10548.823336] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10549.161758] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10554.680989] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10561.020147] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10568.547178] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10568.826388] Lustre: lustre-OST0001: new disk, initializing [10568.834813] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [10568.849926] Lustre: 323900:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10569.008074] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10574.478409] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [10574.487710] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [10574.550627] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10575.359939] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10575.601585] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10575.609841] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10576.686410] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10576.694094] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [10578.730887] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10578.743568] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [10582.883394] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [10582.900228] LustreError: 324203:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [10585.416265] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [10585.617233] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [10590.809981] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9c8590289800.ost_server_uuid 50 [10591.080117] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [10591.090977] LustreError: 324197:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 14 previous similar messages [10591.977935] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9c8590289800.ost_server_uuid in FULL state after 0 sec [10597.133705] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9c8590289800.ost_server_uuid 50 [10599.025667] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9c8590289800.ost_server_uuid in FULL state after 0 sec [10600.383574] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [10615.789161] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10615.791626] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10615.807203] LustreError: Skipped 2 previous similar messages [10615.807969] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10615.820899] Lustre: Skipped 2 previous similar messages [10615.837101] Lustre: Skipped 1 previous similar message [10619.630369] Lustre: server umount lustre-OST0001 complete [10620.906339] LustreError: 322886:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10620.919452] LustreError: 322886:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10626.023315] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10626.030227] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10626.051045] Lustre: Skipped 1 previous similar message [10626.062402] Lustre: Skipped 2 previous similar messages [10631.149235] LustreError: 324204:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10631.176390] LustreError: 324204:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 4 previous similar messages [10631.703181] Lustre: server umount lustre-OST0000 complete [10636.264186] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10636.298427] Lustre: Skipped 1 previous similar message [10636.303513] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10636.317314] Lustre: Skipped 2 previous similar messages [10642.880744] Lustre: server umount lustre-MDT0000 complete [10647.403455] LustreError: 323899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838447 with bad export cookie 9167569215075880800 [10647.404989] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10647.412984] LustreError: 323899:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [10647.802899] Lustre: server umount lustre-MDT0001 complete [10659.277487] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [10661.801342] Key type lgssc unregistered [10662.135236] LNet: 326592:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10662.151509] LNetError: 326592:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10662.180377] LNet: Removed LNI 192.168.206.156@tcp [10663.109720] Key type .llcrypt unregistered [10663.111906] Key type ._llcrypt unregistered [10684.335333] Key type ._llcrypt registered [10684.340051] Key type .llcrypt registered [10684.457247] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [10685.460083] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10685.492079] alg: No test for adler32 (adler32-zlib) [10686.461727] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [10686.627042] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [10688.264491] Key type lgssc registered [10689.199890] Lustre: Echo OBD driver; http://www.lustre.org/ [10695.747263] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10695.768586] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10697.301829] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10700.854310] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10706.404581] LustreError: 328012:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10706.415721] LustreError: 328012:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [10708.041206] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10711.526264] LustreError: 328013:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10713.950238] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10714.256965] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10718.638552] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10727.483205] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10727.859380] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10733.549839] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10739.140266] LustreError: 329691:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10741.021600] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10742.250290] LustreError: 329693:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10742.282047] LustreError: 329693:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10742.302323] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [10747.371478] LustreError: 329694:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10747.400783] LustreError: 329694:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10748.341283] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10748.602369] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [10754.075159] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [10754.979767] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10762.737948] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10769.743870] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [10769.905260] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [10876.897662] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10876.905287] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10876.939659] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [10878.792803] Lustre: server umount lustre-OST0001 complete [10882.019019] LustreError: 329693:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10882.050879] LustreError: 329693:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [10885.603236] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10885.611416] LustreError: Skipped 1 previous similar message [10885.614782] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10885.620989] Lustre: Skipped 1 previous similar message [10885.626987] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10885.632571] Lustre: Skipped 1 previous similar message [10887.138427] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10887.146200] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10890.981372] Lustre: server umount lustre-OST0000 complete [10897.379078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10897.383038] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10897.403416] Lustre: Skipped 1 previous similar message [10897.424146] Lustre: Skipped 2 previous similar messages [10900.777741] Lustre: server umount lustre-MDT0000 complete [10902.497430] LustreError: 331695:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10902.529423] LustreError: 331695:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [10905.156825] LustreError: 331697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838704 with bad export cookie 15202324684538960077 [10905.162641] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10905.180593] LustreError: 331697:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10905.565182] Lustre: server umount lustre-MDT0001 complete [10919.247672] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [10922.316958] Key type lgssc unregistered [10922.608754] LNet: 332959:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10922.618375] LNetError: 332959:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10922.662557] LNet: Removed LNI 192.168.206.156@tcp [10923.721160] Key type .llcrypt unregistered [10923.724216] Key type ._llcrypt unregistered [10972.812411] Key type ._llcrypt registered [10972.815306] Key type .llcrypt registered [10972.932440] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [10974.225539] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10974.369346] alg: No test for adler32 (adler32-zlib) [10975.427280] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [10975.662085] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [10977.456919] Key type lgssc registered [10978.769983] Lustre: Echo OBD driver; http://www.lustre.org/ [10988.705710] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10988.724318] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10990.272371] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10995.466450] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [10999.782024] LustreError: 335267:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10999.813141] LustreError: 335267:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11003.887584] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11004.267254] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11008.034460] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11010.799348] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [11018.498383] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11019.041934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11025.425261] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11029.296994] LustreError: 336757:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11034.410640] LustreError: 336758:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11034.433065] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [11034.434276] LustreError: 336758:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11034.504765] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11034.708904] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11039.739073] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [11040.920475] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11050.649443] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11057.785717] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [11070.436649] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11070.443606] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11070.447064] Lustre: Skipped 1 previous similar message [11075.557841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11075.565530] Lustre: Skipped 6 previous similar messages [11076.098157] Lustre: server umount lustre-MDT0000 complete [11079.851858] LustreError: 335248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787838879 with bad export cookie 8282606246616019492 [11079.857836] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11079.867492] LustreError: 335248:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [11080.379386] Lustre: server umount lustre-MDT0001 complete [11094.915353] Lustre: server umount lustre-OST0000 complete [11097.059606] Lustre: 334530:0:(client.c:2503:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1787838880/real 1787838880] req@ffff8b4011553480 x1874684831395968/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1787838896 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [11097.084141] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11097.097574] Lustre: Skipped 2 previous similar messages [11099.634336] Lustre: server umount lustre-OST0001 complete [11115.877367] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [11127.066680] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11127.463565] LustreError: 340532:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11127.566939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11132.332503] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11132.898082] LustreError: 340533:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11141.024540] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11141.429322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11145.834665] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11148.945740] Lustre: 341673:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [11156.259186] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11162.949256] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11165.870124] LustreError: 342027:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11165.891575] LustreError: 342027:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11171.240061] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11171.507301] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [11171.515297] Lustre: Skipped 1 previous similar message [11172.572348] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [11172.579718] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [11178.651481] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11187.626643] Lustre: DEBUG MARKER: Using TIMEOUT=20 [11192.510791] Lustre: 343543:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [11198.440463] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11198.454792] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11198.471479] Lustre: Skipped 3 previous similar messages [11198.477786] Lustre: Skipped 2 previous similar messages [11203.565867] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11203.574102] Lustre: Skipped 1 previous similar message [11204.511116] Lustre: server umount lustre-MDT0000 complete [11208.450481] LustreError: 343545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839008 with bad export cookie 8282606246616021193 [11208.465730] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11208.468260] LustreError: 343545:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [11208.674706] LustreError: 340528:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11208.680795] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11208.690152] LustreError: 340528:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11208.868808] Lustre: server umount lustre-MDT0001 complete [11215.056637] Lustre: server umount lustre-OST0000 complete [11219.194479] Lustre: server umount lustre-OST0001 complete [11223.197432] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [11224.694056] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [11226.620738] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 09:57:04 (1787839024) [11249.584290] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [11257.243957] loop0: detected capacity change from 0 to 1099511627776 [11257.763480] SGI XFS with ACLs, security attributes, quota, no debug enabled [11257.769428] *** VALIDATE xfs *** [11257.780944] XFS (loop0): Mounting V5 Filesystem [11257.792405] XFS (loop0): Ending clean mount [11258.975853] loop1: detected capacity change from 0 to 18691697672192 [11353.085523] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [11362.018157] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [11391.319958] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11391.531277] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11391.578492] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11391.683809] Lustre: lustre-MDT0000: new disk, initializing [11391.763968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11391.785254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11396.103176] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11404.792439] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11404.891521] Lustre: 348679:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11404.940993] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11404.945031] Lustre: Skipped 1 previous similar message [11405.026281] Lustre: lustre-MDT0001: new disk, initializing [11405.159089] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11405.169996] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [11408.572645] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11416.770831] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11421.741458] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11428.337563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11428.842911] Lustre: lustre-OST0000: new disk, initializing [11428.857513] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [11428.870811] Lustre: 350646:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [11430.196818] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [11430.208748] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [11430.310469] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [11435.350238] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11445.719950] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11449.730861] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [11449.977879] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [11454.615374] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [11454.853617] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [11461.091859] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11461.097304] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11461.106171] Lustre: Skipped 2 previous similar messages [11461.125838] Lustre: Skipped 1 previous similar message [11462.902570] Lustre: server umount lustre-OST0000 complete [11471.342477] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11471.348518] Lustre: Skipped 3 previous similar messages [11472.257753] Lustre: server umount lustre-MDT0000 complete [11475.351753] LustreError: 348670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839274 with bad export cookie 8282606246616022747 [11475.359754] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11475.364738] LustreError: 348670:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11475.700541] Lustre: server umount lustre-MDT0001 complete [11482.137429] XFS (loop0): Unmounting Filesystem [11486.442226] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 10:01:24 (1787839284) [11492.353147] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11492.834750] LustreError: 353213:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11492.855262] LustreError: 353213:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11492.928546] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11492.932202] Lustre: Skipped 2 previous similar messages [11496.178242] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11503.042319] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11507.375437] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11514.577121] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11521.874110] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11528.240039] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11534.605685] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11542.168801] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11553.253056] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11553.264283] Lustre: Skipped 2 previous similar messages [11553.270484] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [11553.272693] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [11553.289759] Lustre: Skipped 1 previous similar message [11556.419844] Lustre: server umount lustre-OST0000 complete [11559.907896] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [11574.242768] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [11574.565083] Lustre: server umount lustre-MDT0000 complete [11577.827413] LustreError: 353213:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11577.864641] LustreError: 353213:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 3 previous similar messages [11578.393114] LustreError: 353948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787839378 with bad export cookie 8282606246616023629 [11578.398627] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11578.405078] LustreError: 353948:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [11578.809308] Lustre: server umount lustre-MDT0001 complete [11590.263238] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [11593.270279] Key type lgssc unregistered [11593.539842] LNet: 356847:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [11593.550820] LNetError: 356847:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [11594.598780] LNet: Removed LNI 192.168.206.156@tcp [11595.761678] Key type .llcrypt unregistered [11595.767402] Key type ._llcrypt unregistered [11623.287202] Key type ._llcrypt registered [11623.289420] Key type .llcrypt registered [11623.421367] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [11624.596335] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [11624.615502] alg: No test for adler32 (adler32-zlib) [11625.688439] Lustre: Lustre: Build Version: 2.17.57_65_g89f8e4a [11625.952427] LNet: Added LNI 192.168.206.156@tcp [8/256/0/180] [11627.624190] Key type lgssc registered [11628.597422] Lustre: Echo OBD driver; http://www.lustre.org/ [11635.400242] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [11635.425973] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11636.794646] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11641.114438] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11646.947808] LustreError: 358417:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11646.971265] LustreError: 358417:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11649.442658] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11649.824444] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11653.372302] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11659.658427] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [11664.446366] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [11670.416620] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [11670.692303] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [11678.491757] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11688.346185] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [11696.097279] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [11696.112194] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11696.125574] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11697.636585] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [11697.639345] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11697.643065] Lustre: Skipped 1 previous similar message [11701.282143] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.56@tcp (stopping) [11701.293039] Lustre: Skipped 1 previous similar message [11702.104399] Lustre: server umount lustre-MDT0001 complete [11702.758044] LustreError: 359506:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [11702.789765] LustreError: 359506:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [11706.402683] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.56@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11711.519763] LustreError: 359708:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.56@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [11711.541122] LustreError: 359708:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [11711.568618] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11711.690391] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [11711.694660] Lustre: Found index 1 for lustre-MDT0001, updating log [11711.740659] Lustre: 358404:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11712.046274] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11716.091889] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [11717.127335] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [11722.242273] LustreError: 362021:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [11722.253571] LustreError: 362021:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [11722.261712] LustreError: 358409:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [11722.342876] LustreError: 362024:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.156@tcp: cfg command failed: rc = -17 [11722.363414] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [11722.363414] [12023.864603] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.56@tcp (stopping) [12024.292252] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12024.301446] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12024.312391] Lustre: Skipped 1 previous similar message [12028.919313] Lustre: server umount lustre-MDT0001 complete [12028.961270] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.56@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12039.210891] LustreError: 364149:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.56@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12039.244972] LustreError: 364149:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12039.750655] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12039.865352] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12039.869579] Lustre: Found index 1 for lustre-MDT0001, updating log [12039.899389] Lustre: 358406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12040.165792] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12044.162152] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [12045.305886] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12046.843963] LustreError: 371016:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12046.851815] LustreError: 371016:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12046.873513] LustreError: 358409:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12046.879394] LustreError: 358409:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12046.940284] LustreError: 371025:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [12046.948171] LustreError: 371025:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [12046.956266] LustreError: 371025:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.156@tcp: cfg command failed: rc = -17 [12046.965481] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.206.156@tcp [12046.965481] [12351.550120] Lustre: lustre-MDT0001: Not available for connect from 192.168.206.56@tcp (stopping) [12351.553382] Lustre: Skipped 2 previous similar messages [12352.483937] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [12352.490928] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12352.524668] Lustre: Skipped 1 previous similar message [12356.285395] Lustre: server umount lustre-MDT0001 complete [12356.645890] LustreError: 358412:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.206.56@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [12356.658142] LustreError: 358412:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 2 previous similar messages [12365.520344] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12365.656872] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [12365.662656] Lustre: Found index 1 for lustre-MDT0001, updating log [12365.726943] Lustre: 358406:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12366.026190] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [12369.770464] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [12370.939290] LustreError: 380238:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [12370.946515] LustreError: 380238:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [12370.955194] LustreError: 358409:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [12370.959327] LustreError: 358409:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [12371.007503] LustreError: 380245:0:(lod_lov.c:208:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [12371.020113] LustreError: 380245:0:(obd_config.c:2102:class_config_llog_handler()) MGC192.168.206.156@tcp: cfg command failed: rc = -114 [12371.029156] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [12371.029156] [12371.444122] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [12688.354611] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12688.367156] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12688.383259] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12688.395025] Lustre: Skipped 2 previous similar messages [12694.381100] Lustre: server umount lustre-MDT0000 complete [12698.014661] LustreError: 358399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840497 with bad export cookie 11665799855131774894 [12698.024834] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12698.037024] LustreError: 358399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12699.105353] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12699.108992] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12699.119150] Lustre: Skipped 2 previous similar messages [12699.134530] LustreError: 358413:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 5 previous similar messages [12704.452761] Lustre: server umount lustre-MDT0001 complete [12709.569870] Lustre: server umount lustre-OST0000 complete [12719.210838] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [12728.141030] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [12766.190122] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 10:22:44 (1787840564) [12773.448592] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12773.647155] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12773.664283] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12773.711520] Lustre: lustre-MDT0000: new disk, initializing [12773.751375] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12773.774739] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12777.488508] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [12786.958991] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12787.085510] Lustre: 391298:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12787.127128] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12787.131487] Lustre: Skipped 1 previous similar message [12787.195361] Lustre: lustre-MDT0001: new disk, initializing [12787.282565] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12787.313839] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12787.319255] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12791.464870] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [12800.156210] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12804.962536] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12810.918962] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12811.153329] Lustre: lustre-OST0000: new disk, initializing [12811.160331] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12811.178835] Lustre: 393267:0:(osd_compat.c:1353:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12811.311407] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12816.344636] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_default_debug -1 all [12817.484186] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12817.498758] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12817.615147] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12824.945825] Lustre: DEBUG MARKER: oleg656-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12833.178870] Lustre: server umount lustre-MDT0000 complete [12833.255098] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12833.274928] Lustre: Skipped 1 previous similar message [12833.277819] LustreError: 391305:0:(ldlm_lib.c:1190:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12833.283841] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12833.285758] LustreError: 391305:0:(ldlm_lib.c:1190:target_handle_connect()) Skipped 1 previous similar message [12836.111149] LustreError: 392105:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [12836.113158] LustreError: 391289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1787840635 with bad export cookie 11665799855131776847 [12836.114960] LustreError: MGC192.168.206.156@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12836.121223] LustreError: 392105:0:(client.c:1394:ptlrpc_import_delay_req()) @@@ IMP_CLOSED req@00000000474f1b1b x1874685513296512/t0(0) o1000->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 264/4320 e 0 to 0 dl 0 ref 2 fl Rpc:QU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [12836.155938] LustreError: 391289:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12836.365720] Lustre: server umount lustre-MDT0001 complete [12850.039485] Lustre: server umount lustre-OST0000 complete [12876.361658] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing set_hostid [12884.622264] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing load_modules_local [12908.440863] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 12634 sec ========= 10:25:06 (1787840706) [12909.923681] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 10:25:07 (1787840707) === [12912.625795] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 10:25:10 (1787840710) === [12956.508340] Lustre: DEBUG MARKER: oleg656-server.virtnet: executing unload_modules_local [12959.765643] Key type lgssc unregistered [12960.045741] LNet: 399633:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12960.054310] LNetError: 399633:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12960.090778] LNet: Removed LNI 192.168.206.156@tcp [12960.840386] Key type .llcrypt unregistered [12960.844539] Key type ._llcrypt unregistered