[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 475111581 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001010] APIC: Switch to symmetric I/O mode setup [ 0.003285] x2apic enabled [ 0.004009] Switched APIC routing to physical x2apic. [ 0.005017] kvm-guest: setup PV IPIs [ 0.007822] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.008000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.008024] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.009015] pid_max: default: 32768 minimum: 301 [ 0.010141] LSM: Security Framework initializing [ 0.011074] Yama: becoming mindful. [ 0.013028] SELinux: Initializing. [ 0.014090] *** VALIDATE selinux *** [ 0.023056] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.027831] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.028143] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.030077] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031104] *** VALIDATE tmpfs *** [ 0.032505] *** VALIDATE proc *** [ 0.033227] *** VALIDATE cgroup *** [ 0.034009] *** VALIDATE cgroup2 *** [ 0.035235] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.037139] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.038009] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.039030] Spectre V2 : User space: Vulnerable [ 0.040009] Speculative Store Bypass: Vulnerable [ 0.043436] debug: unmapping init [mem 0xffffffffbb059000-0xffffffffbb060fff] [ 0.046180] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.047633] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.048024] ... version: 2 [ 0.049016] ... bit width: 48 [ 0.050014] ... generic registers: 4 [ 0.051020] ... value mask: 0000ffffffffffff [ 0.052017] ... max period: 00007fffffffffff [ 0.053018] ... fixed-purpose events: 3 [ 0.054014] ... event mask: 000000070000000f [ 0.055315] rcu: Hierarchical SRCU implementation. [ 0.057480] smp: Bringing up secondary CPUs ... [ 0.058629] x86: Booting SMP configuration: [ 0.059034] .... node #0, CPUs: #1 #2 #3 [ 0.062550] smp: Brought up 1 node, 4 CPUs [ 0.064015] smpboot: Max logical packages: 1 [ 0.065020] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.290050] node 0 deferred pages initialised in 222ms [ 0.294354] devtmpfs: initialized [ 0.296364] x86/mm: Memory block size: 128MB [ 0.299952] gcov: version magic: 0x41383552 [ 0.302345] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.306110] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.309398] pinctrl core: initialized pinctrl subsystem [ 0.312253] [ 0.312867] ************************************************************* [ 0.314012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.316017] ** ** [ 0.319020] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.321017] ** ** [ 0.324019] ** This means that this kernel is built to expose internal ** [ 0.326017] ** IOMMU data structures, which may compromise security on ** [ 0.329024] ** your system. ** [ 0.332022] ** ** [ 0.334021] ** If you see this message and you are not debugging the ** [ 0.336019] ** kernel, report this immediately to your vendor! ** [ 0.339013] ** ** [ 0.341017] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.344016] ************************************************************* [ 0.346839] NET: Registered protocol family 16 [ 0.349537] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.352087] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.355083] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.358126] cpuidle: using governor menu [ 0.359759] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.362709] PCI: Using configuration type 1 for base access [ 0.365167] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.376066] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.377021] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.379061] cryptd: max_cpu_qlen set to 1000 [ 0.381302] ACPI: Added _OSI(Module Device) [ 0.383015] ACPI: Added _OSI(Processor Device) [ 0.384012] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.386014] ACPI: Added _OSI(Processor Aggregator Device) [ 0.391379] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.398123] ACPI: Interpreter enabled [ 0.399077] ACPI: PM: (supports S0 S3 S4 S5) [ 0.401013] ACPI: Using IOAPIC for interrupt routing [ 0.403124] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.406475] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.417183] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.420046] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.423023] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.426088] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.432472] acpiphp: Slot [2] registered [ 0.434125] acpiphp: Slot [5] registered [ 0.435267] acpiphp: Slot [6] registered [ 0.437350] acpiphp: Slot [7] registered [ 0.440227] acpiphp: Slot [8] registered [ 0.441169] acpiphp: Slot [9] registered [ 0.443244] acpiphp: Slot [10] registered [ 0.445215] acpiphp: Slot [3] registered [ 0.447146] acpiphp: Slot [4] registered [ 0.448180] acpiphp: Slot [11] registered [ 0.450158] acpiphp: Slot [12] registered [ 0.452153] acpiphp: Slot [13] registered [ 0.453190] acpiphp: Slot [14] registered [ 0.455136] acpiphp: Slot [15] registered [ 0.457164] acpiphp: Slot [16] registered [ 0.458172] acpiphp: Slot [17] registered [ 0.461189] acpiphp: Slot [18] registered [ 0.463184] acpiphp: Slot [19] registered [ 0.464153] acpiphp: Slot [20] registered [ 0.466163] acpiphp: Slot [21] registered [ 0.468126] acpiphp: Slot [22] registered [ 0.469116] acpiphp: Slot [23] registered [ 0.471130] acpiphp: Slot [24] registered [ 0.472102] acpiphp: Slot [25] registered [ 0.474115] acpiphp: Slot [26] registered [ 0.475119] acpiphp: Slot [27] registered [ 0.477125] acpiphp: Slot [28] registered [ 0.478106] acpiphp: Slot [29] registered [ 0.480135] acpiphp: Slot [30] registered [ 0.482112] acpiphp: Slot [31] registered [ 0.483062] PCI host bridge to bus 0000:00 [ 0.484020] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.487025] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.489026] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.492033] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.494029] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.498039] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.500223] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.504254] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.508563] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.519836] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.525116] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.528023] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.531024] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.534021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.537480] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.539754] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.543061] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.546892] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.553017] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.570017] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.575011] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.580430] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.591016] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.600016] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.616015] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.624000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.632012] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.639017] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.668018] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.678000] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.688014] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.693016] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.710018] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.720015] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.728014] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.734016] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.751015] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.762101] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.770017] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.777015] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.794017] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.805040] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.812016] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.820016] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.846018] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.860128] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.862411] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.865391] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.867355] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.870236] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.875031] iommu: Default domain type: Passthrough [ 0.876415] SCSI subsystem initialized [ 0.878144] ACPI: bus type USB registered [ 0.879082] usbcore: registered new interface driver usbfs [ 0.881089] usbcore: registered new interface driver hub [ 0.882089] usbcore: registered new device driver usb [ 0.884160] pps_core: LinuxPPS API ver. 1 registered [ 0.886012] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.888060] PTP clock support registered [ 0.890262] EDAC MC: Ver: 3.0.0 [ 0.891690] PCI: Using ACPI for IRQ routing [ 0.892521] NetLabel: Initializing [ 0.894012] NetLabel: domain hash size = 128 [ 0.895011] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.897094] NetLabel: unlabeled traffic allowed by default [ 0.899154] vgaarb: loaded [ 0.900273] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.902012] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.908806] clocksource: Switched to clocksource kvm-clock [ 1.010642] VFS: Disk quotas dquot_6.6.0 [ 1.011918] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.014341] *** VALIDATE ramfs *** [ 1.015305] *** VALIDATE hugetlbfs *** [ 1.016509] pnp: PnP ACPI init [ 1.018927] pnp: PnP ACPI: found 6 devices [ 1.039637] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.042197] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.043847] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.045602] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.048410] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.050865] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.053430] NET: Registered protocol family 2 [ 1.055705] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.060817] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.064724] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.069762] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.073403] TCP: Hash tables configured (established 65536 bind 65536) [ 1.076557] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.079304] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.082306] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.085092] NET: Registered protocol family 1 [ 1.088176] RPC: Registered named UNIX socket transport module. [ 1.090457] RPC: Registered udp transport module. [ 1.092138] RPC: Registered tcp transport module. [ 1.093935] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.096092] NET: Registered protocol family 44 [ 1.097606] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.100144] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.102573] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.104899] PCI: CLS 0 bytes, default 64 [ 1.106972] Unpacking initramfs... [ 2.600990] debug: unmapping init [mem 0xffff88d9bcc54000-0xffff88d9bffbffff] [ 2.605618] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.608646] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.612260] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.158246] Initialise system trusted keyrings [ 3.161305] Key type blacklist registered [ 3.163536] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.174154] zbud: loaded [ 3.177734] *** VALIDATE nfs *** [ 3.179236] *** VALIDATE nfs4 *** [ 3.181136] pstore: using deflate compression [ 3.185599] Platform Keyring initialized [ 3.299435] NET: Registered protocol family 38 [ 3.301526] Key type asymmetric registered [ 3.303248] Asymmetric key parser 'x509' registered [ 3.305399] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.308922] io scheduler mq-deadline registered [ 3.310862] io scheduler kyber registered [ 3.312598] io scheduler bfq registered [ 3.314757] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.317870] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.320883] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.324213] ACPI: Power Button [PWRF] [ 3.330896] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.338034] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.355144] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.362728] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.381802] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.409948] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.438075] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.442984] Non-volatile memory driver v1.3 [ 3.444889] Linux agpgart interface v0.103 [ 3.477577] virtio_blk virtio1: [vda] 145896 512-byte logical blocks (74.7 MB/71.2 MiB) [ 3.482118] vda: detected capacity change from 0 to 74698752 [ 3.511900] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.515374] vdb: detected capacity change from 0 to 1073741824 [ 3.533275] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.536450] vdc: detected capacity change from 0 to 2621440000 [ 3.555680] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.558196] vdd: detected capacity change from 0 to 2621440000 [ 3.576315] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.580292] vde: detected capacity change from 0 to 4294967296 [ 3.596411] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.599693] vdf: detected capacity change from 0 to 4294967296 [ 3.608316] libphy: Fixed MDIO Bus: probed [ 3.614750] usbcore: registered new interface driver usbserial_generic [ 3.617274] usbserial: USB Serial support registered for generic [ 3.620173] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.624956] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.627417] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.630173] mousedev: PS/2 mouse device common for all mice [ 3.632603] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.634783] rtc_cmos 00:05: RTC can wake from S4 [ 3.640854] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.643625] rtc_cmos 00:05: registered as rtc0 [ 3.648492] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.650430] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.654494] intel_pstate: CPU model not supported [ 3.657048] hid: raw HID events driver (C) Jiri Kosina [ 3.664092] usbcore: registered new interface driver usbhid [ 3.665820] usbhid: USB HID core driver [ 3.667153] drop_monitor: Initializing network drop monitor service [ 3.668936] Initializing XFRM netlink socket [ 3.670579] NET: Registered protocol family 10 [ 3.673789] Segment Routing with IPv6 [ 3.675637] NET: Registered protocol family 17 [ 3.677958] mpls_gso: MPLS GSO support [ 3.684585] RAS: Correctable Errors collector initialized. [ 3.687477] AVX version of gcm_enc/dec engaged. [ 3.689285] AES CTR mode by8 optimization enabled [ 3.779318] sched_clock: Marking stable (3779269777, 0)->(4720599058, -941329281) [ 3.783954] registered taskstats version 1 [ 3.785933] Loading compiled-in X.509 certificates [ 3.788490] zswap: loaded using pool lzo/zbud [ 3.818038] Key type big_key registered [ 3.830370] Key type encrypted registered [ 3.831848] ima: No TPM chip found, activating TPM-bypass! [ 3.833915] ima: Allocated hash algorithm: sha1 [ 3.836348] ima: No architecture policies found [ 3.838617] evm: Initialising EVM extended attributes: [ 3.841035] evm: security.selinux [ 3.842139] evm: security.ima [ 3.843055] evm: security.capability [ 3.844300] evm: HMAC attrs: 0x1 [ 3.847094] rtc_cmos 00:05: setting system clock to 2026-08-14 06:11:39 UTC (1786687899) [ 3.852846] debug: unmapping init [mem 0xffffffffbc003000-0xffffffffbc1fffff] [ 3.855527] debug: unmapping init [mem 0xffffffffbad82000-0xffffffffbb058fff] [ 3.864106] Write protecting the kernel read-only data: 28672k [ 3.867405] debug: unmapping init [mem 0xffffffffb9403000-0xffffffffb95fffff] [ 3.869811] debug: unmapping init [mem 0xffffffffb9d14000-0xffffffffb9dfffff] [ 3.906567] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.913884] systemd[1]: Detected virtualization kvm. [ 3.915579] systemd[1]: Detected architecture x86-64. [ 3.917179] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.946526] systemd[1]: No hostname configured. [ 3.948404] systemd[1]: Set hostname to . [ 3.950351] random: systemd: uninitialized urandom read (16 bytes read) [ 3.953078] systemd[1]: Initializing machine ID from random generator. [ 4.019145] random: ln: uninitialized urandom read (6 bytes read) [ 4.107393] random: systemd: uninitialized urandom read (16 bytes read) [ 4.110515] systemd[1]: Reached target Local File Systems. [ OK ] Reached target Local File Systems. [ 4.116623] systemd[1]: Reached target Initrd Root Device. [ OK ] Reached target Initrd Root Device. [ 4.122663] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. Starting Setup Virtual Console... [ OK ] Started Memstrack Anylazing Service. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Swap. Starting Apply Kernel Variables... [ OK ] Reached target Timers. [ OK ] Listening on Journal Socket (/dev/log). Starting Create Volatile Files and Directories... [ OK ] Reached target Slices. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. [ OK ] Reached target Sockets. Starting Journal Service... Starting Create list of required st…ce nodes for the current kernel... [ OK ] Started Setup Virtual Console. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.811276] device-mapper: uevent: version 1.0.3 [ 4.813747] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.660936] random: fast init done [ 5.675933] virtio_net virtio0 ens2: renamed from eth0 [ 5.738290] scsi host0: ata_piix [ 5.787815] scsi host1: ata_piix [ 5.808492] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.811280] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.509901] dracut-initqueue[583]: RTNETLINK answers: File exists [ 10.351694] random: crng init done [ 10.354907] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. Mounting /sysroot... [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. [ 10.975364] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Timers. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped target System Initialization. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Control Socket. [ OK ] Closed udev Kernel Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.256434] printk: systemd: 26 output lines suppressed due to ratelimiting [ 12.544760] SELinux: Disabled at runtime. [ 12.608660] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 12.617392] systemd[1]: Detected virtualization kvm. [ 12.619516] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.209406] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.213227] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.217758] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.222230] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.228306] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.237070] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.241681] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Mounting Kernel Debug File System... [ OK ] Created slice system-sshd\x2dkeygen.slice. Starting Apply Kernel Variables... [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on initctl Compatibility Named Pipe. Mounting POSIX Message Queue File System... [ OK ] Reached target rpc_pipefs.target. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Stopped target Switch Root. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-getty.slice. Starting Remount Root and Kernel File Systems... [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Paths. [ OK ] Listening on udev Kernel Socket. Activating swap /dev/disk/by-label/SWAP... [ OK ] Stopped target Initrd Root File System. Mounting Huge Pages File System... Starting Create list of required[ 13.394133] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS st…ce nodes for the current kernel... [ OK ] Created slice User and Session Slice. [ OK ] Reached target Slices. [ OK ] Listening on udev Control Socket. Starting udev Coldplug all Devices... [ OK ] Reached target RPC Port Mapper. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Started Apply Kernel Variables. [ OK ] Mounted POSIX Message Queue File System. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ OK ] Activated swap /dev/disk/by-label/SWAP. [ OK ] Mounted Huge Pages File System. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Reached target Swap. Starting Configure read-only root support... Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. Starting udev Kernel Device Manager... [ OK ] Reached target Local File Systems (Pre). Mounting /mnt... Mounting /home/green/git/lustre-release... [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /mnt. [ 13.762857] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.022485] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.146604] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.295926] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.307419] EDAC sbridge: Ver: 1.1.2 [ 16.317082] Key type dns_resolver registered [ 16.622715] NFS: Registering the id_resolver key type [ 16.624807] Key type id_resolver registered [ 16.626362] Key type id_legacy registered [ OK ] Started Configure read-only root support. Starting Load/Save Random Seed... [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Create Volatile Files and Directories... [ OK ] Started Load/Save Random Seed. [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started irqbalance daemon. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Started D-Bus System Message Bus. Starting Network Manager... [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Reached target Timers. [ OK ] Reached target sshd-keygen.target. Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Login Service. [ OK ] Started Network Manager. [ OK ] Reached target Network. Starting Dynamic System Tuning Daemon... Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Network Manager Wait Online... [ OK ] Started GSSAPI Proxy Daemon. Starting Hostname Service... [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Started Command Scheduler. [ OK ] Started Serial Getty on ttyS1. [ OK ] Reached target Login Prompts. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting Notify NFS peers of a restart... Starting System Logging Service... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg409-server login: [ 36.263252] hrtimer: interrupt took 2435933 ns [ 72.146569] libcfs: loading out-of-tree module taints kernel. [ 72.202706] Key type ._llcrypt registered [ 72.205578] Key type .llcrypt registered [ 72.301857] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 93.088298] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 94.982851] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 95.001483] alg: No test for adler32 (adler32-zlib) [ 96.479602] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 97.360041] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 99.239284] Key type lgssc registered [ 101.156098] Lustre: Echo OBD driver; http://www.lustre.org/ [ 119.245904] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 163.478433] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 176.440631] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 176.474337] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 177.852742] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 177.898543] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 178.021498] Lustre: lustre-MDT0000: new disk, initializing [ 178.123083] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 178.147455] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 182.568425] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 196.090131] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 196.210636] Lustre: 6517:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 196.248619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 196.252362] Lustre: Skipped 1 previous similar message [ 196.328533] Lustre: lustre-MDT0001: new disk, initializing [ 196.400592] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 196.426390] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 196.432623] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 200.292460] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 205.184450] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 215.110745] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 215.316546] Lustre: lustre-OST0000: new disk, initializing [ 215.319683] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 215.323958] Lustre: 8453:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 215.376521] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 219.357454] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 219.364845] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 219.479093] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 221.791699] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 235.968349] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 236.102579] Lustre: lustre-OST0001: new disk, initializing [ 236.105315] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 236.109444] Lustre: 9527:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 236.174509] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 242.405249] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 244.779559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 244.796467] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 244.860481] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 253.299208] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 258.329609] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 264.546552] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing check_logdir /tmp/testlogs/ [ 269.795853] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing yml_node [ 275.328975] Lustre: DEBUG MARKER: Client: 2.17.57.1 [ 277.746424] Lustre: DEBUG MARKER: MDS: 2.17.57.1 [ 280.353246] Lustre: DEBUG MARKER: OSS: 2.17.57.1 [ 281.980979] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Fri Aug 14 02:16:15 EDT 2026 [ 302.046890] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 304.308460] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 321.506261] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 321.508445] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 321.525368] Lustre: Skipped 3 previous similar messages [ 321.531018] Lustre: Skipped 2 previous similar messages [ 326.627237] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 326.638723] Lustre: Skipped 4 previous similar messages [ 326.766877] Lustre: server umount lustre-MDT0000 complete [ 330.760696] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688226 with bad export cookie 9385469922177798758 [ 330.768598] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 330.775857] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 331.744478] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 331.752617] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 331.756583] Lustre: Skipped 1 previous similar message [ 331.769183] Lustre: Skipped 1 previous similar message [ 335.776130] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 335.778823] Lustre: Skipped 1 previous similar message [ 337.135168] Lustre: server umount lustre-MDT0001 complete [ 340.814666] Lustre: server umount lustre-OST0000 complete [ 344.552963] Lustre: server umount lustre-OST0001 complete [ 350.926333] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 359.335522] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 398.471564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 398.687878] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 398.720787] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 398.788625] Lustre: lustre-MDT0000: new disk, initializing [ 398.839911] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 398.856301] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 403.508368] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 415.397373] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 415.539379] Lustre: 15086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 415.589387] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 415.601795] Lustre: Skipped 1 previous similar message [ 415.694436] Lustre: lustre-MDT0001: new disk, initializing [ 415.796424] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 415.836508] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 415.854937] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 420.425659] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 429.297406] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 434.691086] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 441.292513] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 441.498871] Lustre: lustre-OST0000: new disk, initializing [ 441.505673] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 441.512793] Lustre: 17056:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 441.625670] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 443.372863] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 443.379791] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 443.547806] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 447.866508] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 456.702291] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 461.112359] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 461.332677] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 465.712406] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 465.925791] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 468.963458] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 468.969442] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 468.998910] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 469.478179] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 479.717177] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 479.734820] Lustre: Skipped 5 previous similar messages [ 482.276202] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 482.473524] Lustre: server umount lustre-OST0000 complete [ 487.394372] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 487.404098] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 487.412247] Lustre: Skipped 1 previous similar message [ 493.445215] Lustre: server umount lustre-MDT0000 complete [ 495.080291] LustreError: 15097:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 495.091248] LustreError: 15097:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 496.961860] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688392 with bad export cookie 9385469922177800291 [ 496.970230] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 496.979289] LustreError: 15078:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 497.410758] Lustre: server umount lustre-MDT0001 complete [ 502.822490] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 02:19:56 (1786688396) [ 508.984564] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 509.332819] LustreError: 19322:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 509.416781] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 514.023514] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 514.541325] LustreError: 19323:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 519.654295] LustreError: 19322:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 523.096579] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 528.384802] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 536.199832] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 542.604205] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 550.373943] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 556.847209] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 565.450769] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 572.219431] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 579.563547] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 579.568840] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 579.583073] Lustre: Skipped 3 previous similar messages [ 579.619069] Lustre: Skipped 5 previous similar messages [ 593.376880] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 593.664294] Lustre: server umount lustre-MDT0000 complete [ 594.923509] LustreError: 21067:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 594.936927] LustreError: 21067:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 597.765144] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688493 with bad export cookie 9385469922177801173 [ 597.770910] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 597.775268] LustreError: 19303:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 598.154893] Lustre: server umount lustre-MDT0001 complete [ 613.012126] Lustre: server umount lustre-OST0000 complete [ 635.496200] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 645.213807] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 645.689614] LustreError: 23703:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 645.762218] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 645.767573] Lustre: Skipped 2 previous similar messages [ 650.921893] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 659.059827] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 663.867860] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 666.729094] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 672.885630] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 679.573417] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 687.780122] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 687.888799] Lustre: lustre-OST0001: new disk, initializing [ 687.892321] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 687.897661] Lustre: 26019:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 689.762149] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 689.771469] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 689.882973] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 693.940982] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 702.623407] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 706.100971] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 720.865175] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 720.873369] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 720.891161] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 720.894850] Lustre: Skipped 6 previous similar messages [ 724.857602] Lustre: server umount lustre-MDT0000 complete [ 728.033801] LustreError: 24723:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 728.055967] LustreError: 24723:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 728.729628] LustreError: 23685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688624 with bad export cookie 9385469922177802020 [ 728.741493] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 728.752108] LustreError: 23685:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 729.151698] Lustre: server umount lustre-MDT0001 complete [ 742.465512] Lustre: server umount lustre-OST0000 complete [ 757.106709] Lustre: server umount lustre-OST0001 complete [ 775.161544] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 786.252514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 786.528073] LustreError: 29036:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 786.637368] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 786.647932] Lustre: Skipped 3 previous similar messages [ 791.972646] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 800.976268] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 806.169736] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 809.388987] Lustre: 30183:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 816.770679] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 823.979835] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 832.808030] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 839.822379] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 848.068161] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 851.793210] Lustre: 32062:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 858.593476] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 858.605821] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 858.626910] Lustre: Skipped 6 previous similar messages [ 858.635693] Lustre: Skipped 6 previous similar messages [ 864.301303] Lustre: server umount lustre-MDT0000 complete [ 868.347589] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688763 with bad export cookie 9385469922177803315 [ 868.349698] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 868.364654] LustreError: 29018:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 868.823932] Lustre: server umount lustre-MDT0001 complete [ 883.460474] Lustre: server umount lustre-OST0000 complete [ 885.199292] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786688764/real 1786688764] req@ffff88d9037faa00 x1873478148194304/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786688780 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 887.594927] Lustre: server umount lustre-OST0001 complete [ 897.509763] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 02:26:31 (1786688791) [ 926.506518] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 936.251812] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 936.479742] LustreError: 34867:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 936.501818] LustreError: 34867:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 941.163512] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 949.878646] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 954.528421] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 957.287773] Lustre: 36015:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 963.802088] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 970.056928] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 978.203320] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 984.396866] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 992.238090] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 995.968801] Lustre: 37897:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1001.445422] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1001.454981] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1001.464328] Lustre: Skipped 1 previous similar message [ 1001.468454] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1001.471651] Lustre: Skipped 5 previous similar messages [ 1006.751179] Lustre: server umount lustre-MDT0000 complete [ 1010.065731] LustreError: 34849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786688905 with bad export cookie 9385469922177804421 [ 1010.072696] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1010.079205] LustreError: 34849:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1010.697929] Lustre: server umount lustre-MDT0001 complete [ 1024.929225] Lustre: server umount lustre-OST0000 complete [ 1028.063880] Lustre: 3649:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786688907/real 1786688907] req@ffff88da3bc82a00 x1873478148240384/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786688923 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1028.415063] Lustre: server umount lustre-OST0001 complete [ 1037.522530] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 02:28:51 (1786688931) [ 1056.948457] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1068.911818] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1069.452494] LustreError: 40039:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1069.481566] LustreError: 40039:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 1069.591052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1069.602520] Lustre: Skipped 7 previous similar messages [ 1074.842353] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1084.306564] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1089.621798] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1092.080752] Lustre: 41190:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1098.026756] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1104.106690] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1113.178325] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1120.025595] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1128.578067] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1133.177590] Lustre: 43067:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 1134.366797] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 1134.374836] Lustre: Skipped 1 previous similar message [ 1140.201966] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1140.221724] Lustre: Skipped 6 previous similar messages [ 1143.802736] Lustre: server umount lustre-MDT0000 complete [ 1147.648073] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786689043 with bad export cookie 9385469922177805527 [ 1147.651824] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1147.654220] LustreError: 40021:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 1154.183380] Lustre: server umount lustre-MDT0001 complete [ 1163.566708] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1172.964078] LustreError: 3648:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff88d903685180 x1873478148292608/t0(0) o250->MGC192.168.204.109@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1178.646611] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1182.706454] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1189.210190] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1194.860808] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1195.004328] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 1195.025340] Lustre: Skipped 1 previous similar message [ 1203.253685] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1205.610872] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1213.425342] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1215.524283] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 1225.697433] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1228.883779] Lustre: server umount lustre-MDT0000 complete [ 1233.407405] LustreError: 40019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786689129 with bad export cookie 9385469922177806654 [ 1233.415550] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1233.427771] LustreError: 40019:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1233.922271] Lustre: server umount lustre-MDT0001 complete [ 1248.396798] Lustre: server umount lustre-OST0000 complete [ 1252.063769] Lustre: 3650:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786689131/real 1786689131] req@ffff88da1f945180 x1873478148331008/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786689147 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 1252.838943] Lustre: server umount lustre-OST0001 complete [ 1262.242539] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 02:32:36 (1786689156) [ 1278.791912] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1290.135759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1296.386532] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1305.969288] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1312.090415] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1315.801331] Lustre: 48832:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 1323.483317] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1331.498922] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1332.071910] LustreError: 49188:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1332.098109] LustreError: 49188:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 34 previous similar messages [ 1340.448521] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1348.048669] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1356.550689] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 1360.840186] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 1373.160016] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1373.167075] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1373.185862] Lustre: Skipped 8 previous similar messages [ 1373.196913] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1373.205066] Lustre: Skipped 19 previous similar messages [ 1374.518280] Lustre: server umount lustre-MDT0000 complete [ 1377.606281] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786689273 with bad export cookie 9385469922177807445 [ 1377.616845] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1377.623847] LustreError: 47661:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1378.019188] Lustre: server umount lustre-MDT0001 complete [ 1392.224900] Lustre: server umount lustre-OST0000 complete [ 1406.224419] Lustre: server umount lustre-OST0001 complete [ 1416.021790] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 02:35:09 (1786689309) [ 1418.434416] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 1420.449985] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 02:35:14 (1786689314) [ 1434.155923] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 1445.390843] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1469.811514] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1470.169872] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1470.302362] Lustre: lustre-MDT0000: new disk, initializing [ 1470.419719] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1475.704913] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1485.770666] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1493.636030] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1493.878854] Lustre: lustre-OST0000: new disk, initializing [ 1493.883142] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1493.886646] Lustre: Skipped 1 previous similar message [ 1493.901921] Lustre: 54909:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1495.367369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 1495.388456] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 1495.427260] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 1500.674792] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1510.765068] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1513.249362] Lustre: DEBUG MARKER: create test files [ 1551.843912] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1554.130862] Lustre: server umount lustre-OST0000 complete [ 1558.515958] Lustre: server umount lustre-MDT0000 complete [ 1572.301500] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 1576.310479] Key type lgssc unregistered [ 1576.814936] LNet: 56542:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1576.821200] LNetError: 56542:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1577.897887] LNet: Removed LNI 192.168.204.109@tcp [ 1579.218631] Key type .llcrypt unregistered [ 1579.221851] Key type ._llcrypt unregistered [ 1608.528066] Key type ._llcrypt registered [ 1608.529865] Key type .llcrypt registered [ 1608.627133] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1609.593532] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1609.702100] alg: No test for adler32 (adler32-zlib) [ 1610.697590] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 1610.963616] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 1612.607192] Key type lgssc registered [ 1613.779933] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1621.218286] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1621.256277] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1622.632932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1627.276269] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1635.200317] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1642.362862] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1642.689934] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1647.862694] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:97 to 0x240000400:129) [ 1648.017337] Lustre: 59277:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 510 < left 544, rollback = 9 [ 1648.037577] Lustre: 59277:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/5, destroy: 0/0/0 [ 1648.050762] Lustre: 59277:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1648.065963] Lustre: 59277:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1648.083369] Lustre: 59277:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1648.094635] Lustre: 59277:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1650.139691] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1659.333968] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1665.001550] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 1671.703378] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 1704.416990] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1704.444880] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1704.465487] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1709.465512] Lustre: server umount lustre-OST0000 complete [ 1714.044669] Lustre: server umount lustre-MDT0000 complete [ 1725.143389] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 1727.939627] Key type lgssc unregistered [ 1728.293991] LNet: 60913:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1728.301845] LNetError: 60913:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1728.318729] LNet: Removed LNI 192.168.204.109@tcp [ 1729.170710] Key type .llcrypt unregistered [ 1729.172480] Key type ._llcrypt unregistered [ 1763.235317] Key type ._llcrypt registered [ 1763.240304] Key type .llcrypt registered [ 1763.383284] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1765.030510] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1765.094817] alg: No test for adler32 (adler32-zlib) [ 1766.277981] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 1766.575317] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 1768.280559] Key type lgssc registered [ 1769.387210] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1777.238960] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1777.267258] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1778.700305] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1783.037444] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1789.820443] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1796.922673] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1797.312993] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1803.447165] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 1803.527200] Lustre: 63743:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 1803.541392] Lustre: 63743:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 1803.554672] Lustre: 63743:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 1803.564022] Lustre: 63743:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 1803.567932] Lustre: 63743:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 1803.571837] Lustre: 63743:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 1803.753667] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1811.304889] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1816.387632] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 1854.946266] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1854.959985] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1860.067654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1860.444268] Lustre: server umount lustre-OST0000 complete [ 1864.655387] Lustre: server umount lustre-MDT0000 complete [ 1877.987761] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 1880.651440] Key type lgssc unregistered [ 1880.931703] LNet: 65329:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1880.937882] LNetError: 65329:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1880.968034] LNet: Removed LNI 192.168.204.109@tcp [ 1881.885922] Key type .llcrypt unregistered [ 1881.890239] Key type ._llcrypt unregistered [ 1902.855152] Key type ._llcrypt registered [ 1902.858747] Key type .llcrypt registered [ 1903.047555] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 1921.534197] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 1923.198456] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1923.256833] alg: No test for adler32 (adler32-zlib) [ 1924.412691] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 1924.732282] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 1926.407180] Key type lgssc registered [ 1927.682558] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1970.630534] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1970.682588] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1971.927600] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1971.960341] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1972.111503] Lustre: lustre-MDT0000: new disk, initializing [ 1972.273290] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1972.307153] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1977.104828] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 1988.072039] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1988.198132] Lustre: 68487:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1988.252767] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1988.257391] Lustre: Skipped 1 previous similar message [ 1988.333645] Lustre: lustre-MDT0001: new disk, initializing [ 1988.473133] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1988.500856] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1988.513384] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1993.770692] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2003.459394] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2009.073959] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2015.727439] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2016.002391] Lustre: lustre-OST0000: new disk, initializing [ 2016.006891] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2016.014537] Lustre: 70455:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2016.137225] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2022.354465] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2024.038901] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2024.050448] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2024.159284] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2032.020787] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2037.317585] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2037.582675] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2042.573038] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2042.800383] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2044.900019] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2044.903335] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2044.906501] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2044.914369] Lustre: Skipped 1 previous similar message [ 2050.019401] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2050.034433] Lustre: Skipped 2 previous similar messages [ 2051.017920] Lustre: server umount lustre-OST0000 complete [ 2055.141185] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2055.145981] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2055.147423] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2055.170775] Lustre: Skipped 1 previous similar message [ 2060.256914] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2060.267198] Lustre: Skipped 2 previous similar messages [ 2061.426473] Lustre: server umount lustre-MDT0000 complete [ 2065.356463] LustreError: 68479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786689961 with bad export cookie 8969651127657244394 [ 2065.372496] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2065.379695] LustreError: 68479:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2066.068389] Lustre: server umount lustre-MDT0001 complete [ 2077.422615] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 02:46:10 (1786689970) [ 2090.489264] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 2135.926410] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 2147.412683] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 2191.558593] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2191.896591] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2191.929535] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2192.026285] Lustre: lustre-MDT0000: new disk, initializing [ 2192.119243] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2192.147396] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2198.047541] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2211.100145] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2211.240480] Lustre: 76479:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2211.316423] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2211.320758] Lustre: Skipped 1 previous similar message [ 2211.430527] Lustre: lustre-MDT0001: new disk, initializing [ 2211.570243] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2211.606584] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2211.617416] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2217.257294] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2229.043280] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2236.949009] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2244.710683] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2245.140898] Lustre: lustre-OST0000: new disk, initializing [ 2245.143969] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2245.148427] Lustre: 78452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2245.223156] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2250.782344] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2250.799130] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2250.905983] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2251.826438] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2264.363417] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2270.515751] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2270.752154] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2276.006931] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2276.269245] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2281.952329] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2281.954486] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2281.960890] LustreError: Skipped 1 previous similar message [ 2281.961557] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2281.997341] Lustre: Skipped 1 previous similar message [ 2284.974295] Lustre: server umount lustre-OST0000 complete [ 2292.197513] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2292.204789] Lustre: Skipped 1 previous similar message [ 2292.205576] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2292.225207] Lustre: Skipped 2 previous similar messages [ 2296.522851] Lustre: server umount lustre-MDT0000 complete [ 2301.501740] LustreError: 76472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786690197 with bad export cookie 8969651127657245276 [ 2301.503260] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2301.513243] LustreError: 76472:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [ 2301.789308] Lustre: server umount lustre-MDT0001 complete [ 2313.374717] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 02:50:07 (1786690207) [ 2321.051789] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2321.725891] LustreError: 80869:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2321.755171] LustreError: 80869:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 5 previous similar messages [ 2321.844313] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2327.015134] LustreError: 80870:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2327.248875] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2332.128143] LustreError: 80869:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2337.248257] LustreError: 80870:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2337.589521] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2342.836420] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2351.063953] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2358.334363] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2366.048628] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2366.526945] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2366.533674] Lustre: Skipped 1 previous similar message [ 2373.909257] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2382.648778] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2387.008977] Lustre: 83542:0:(genops.c:1795:obd_export_evict_by_uuid()) MGS: evicting 68ab83a1-9d55-461a-b2db-419d8c444d02 at adminstrative request [ 2387.958836] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2387.990077] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 2398.188545] Lustre: MGS: Client 68ab83a1-9d55-461a-b2db-419d8c444d02 (at 0@lo) reconnecting [ 2398.216739] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x7c7a9a2d67312dce to 0x7c7a9a2d6731306e [ 2398.239445] Lustre: MGC192.168.204.109@tcp: Connection restored to 0@lo (at 0@lo) [ 2429.859815] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2430.063754] Lustre: lustre-OST0001: new disk, initializing [ 2430.068946] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2430.081296] Lustre: 83939:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2437.871054] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2437.903275] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2438.065075] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2441.086812] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2452.504634] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2458.604070] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2458.613734] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2458.639155] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 2462.739130] Lustre: server umount lustre-OST0001 complete [ 2463.744703] LustreError: 82691:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2463.764548] LustreError: 82691:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2468.834530] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2468.857400] Lustre: Skipped 2 previous similar messages [ 2473.926791] Lustre: server umount lustre-OST0000 complete [ 2485.095647] Lustre: server umount lustre-MDT0000 complete [ 2489.318928] LustreError: 80870:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2489.355715] LustreError: 80870:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2489.448835] LustreError: 80851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786690385 with bad export cookie 8969651127657246830 [ 2489.451163] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2489.460616] LustreError: 80851:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2490.167874] Lustre: server umount lustre-MDT0001 complete [ 2502.128711] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 02:53:15 (1786690395) [ 2504.396838] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 2506.906928] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 02:53:20 (1786690400) [ 2508.712989] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 2511.914054] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 02:53:24 (1786690404) [ 2514.655550] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 2517.250866] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 02:53:30 (1786690410) [ 2519.032784] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 2521.538917] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 02:53:34 (1786690414) [ 2524.209672] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 2532.151799] loop: module loaded [ 2532.200138] loop0: detected capacity change from 0 to 4294967296 [ 2532.312633] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 2534.346525] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 2538.485884] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 2542.510030] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 2572.582214] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2572.640988] LustreError: 88063:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 2572.649727] LustreError: 88063:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 2572.655304] LustreError: 88063:0:(tgt_mount.c:2579:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 2572.680719] LustreError: 88063:0:(super25.c:179:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 2593.931928] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 2608.669951] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 2652.531297] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2652.823846] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2652.878966] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2652.986257] Lustre: lustre-MDT0000: new disk, initializing [ 2653.067625] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2653.074977] Lustre: Skipped 1 previous similar message [ 2653.098148] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2657.992810] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2670.613760] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2670.704499] Lustre: 90951:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2671.093491] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2676.524240] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2688.070664] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2695.036114] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2703.903666] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2704.163174] Lustre: 92917:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2711.376918] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2712.657251] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2712.850382] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2722.132139] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2727.458670] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2727.806676] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2733.383715] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2733.640576] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2738.671026] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2738.676596] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2738.690278] LustreError: Skipped 2 previous similar messages [ 2738.704446] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2738.717876] Lustre: Skipped 3 previous similar messages [ 2738.740578] Lustre: Skipped 8 previous similar messages [ 2741.930851] Lustre: server umount lustre-OST0000 complete [ 2747.886320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2752.731827] Lustre: server umount lustre-MDT0000 complete [ 2754.026147] LustreError: 90961:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2756.937664] LustreError: 90943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786690652 with bad export cookie 8969651127657247446 [ 2756.942993] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2756.950840] LustreError: 90943:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2757.390542] Lustre: server umount lustre-MDT0001 complete [ 2767.014129] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 02:57:40 (1786690660) [ 2774.652191] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2781.417350] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2789.876725] LustreError: 95331:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2789.901728] LustreError: 95331:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 2792.758265] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2793.099340] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-60 [ 2793.103520] Lustre: Skipped 3 previous similar messages [ 2797.354156] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2805.320673] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2810.913435] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2818.061598] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2824.844544] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2833.313830] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2838.349542] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2838.650444] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2845.562994] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2845.932480] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2854.620339] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2854.833115] Lustre: lustre-OST0001: new disk, initializing [ 2854.835767] Lustre: Skipped 2 previous similar messages [ 2854.839700] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2854.846201] Lustre: Skipped 3 previous similar messages [ 2854.849859] Lustre: 98701:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2862.513973] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2863.802297] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 2863.816265] Lustre: Skipped 2 previous similar messages [ 2863.837514] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 2863.941200] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 2872.875583] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2878.700141] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 2879.112376] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2884.938423] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 2885.220714] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2906.503709] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 2935.839287] Lustre: Failing over lustre-MDT0000 [ 2936.290050] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2936.295836] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2936.306589] Lustre: Skipped 2 previous similar messages [ 2936.312962] LustreError: 96059:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2936.342059] Lustre: server umount lustre-MDT0000 complete [ 2957.279128] Lustre: 66709:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786690836/real 1786690836] req@00000000d388de85 x1873480064985984/t0(0) o400->MGC192.168.204.109@tcp@0@lo:26/25 lens 224/224 e 0 to 1 dl 1786690852 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2957.301423] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2962.638368] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 2962.642267] LDISKFS-fs (dm-0): recovery complete [ 2962.649800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2966.497721] LustreError: 66706:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@00000000b0467d64 x1873480064994048/t0(0) o250->MGC192.168.204.109@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:4294967295 [ 2967.083524] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 2971.432370] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 2971.442469] Lustre: 95328:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2972.156722] Lustre: 96059:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 2972.167220] Lustre: lustre-MDT0000-lwp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 2972.175288] Lustre: 96059:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 1 previous similar message [ 2972.372745] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2972.449320] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 2972.719384] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2973.043348] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2973.069151] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2973.407119] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2973.434024] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 1 previous similar message [ 2973.461973] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2973.476585] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 7 previous similar messages [ 2974.140528] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2974.167392] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2974.496142] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2974.510798] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 2 previous similar messages [ 2975.504367] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2975.516724] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 5 previous similar messages [ 2976.160183] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2976.186928] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2976.520156] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2976.530021] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 5 previous similar messages [ 2979.761997] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2979.779871] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 12 previous similar messages [ 2980.417996] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2980.431210] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2980.759120] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2980.769622] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 12 previous similar messages [ 2987.816386] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 2987.833461] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 24 previous similar messages [ 2988.465295] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 2988.485482] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 2988.807242] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 2988.820649] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 24 previous similar messages [ 3004.082624] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 3004.100624] Lustre: 101567:0:(ldlm_lib.c:2085:extend_recovery_timer()) Skipped 50 previous similar messages [ 3004.750174] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 3004.770477] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3005.104447] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 3005.111854] LustreError: 101567:0:(ldlm_lib.c:2698:replay_request_or_update()) Skipped 50 previous similar messages [ 3031.215179] Lustre: 101567:0:(ldlm_lib.c:2394:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 3031.230442] Lustre: 101567:0:(ldlm_lib.c:2404:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 3031.245308] Lustre: 101567:0:(genops.c:1622:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 321ee7c0-3964-49cd-9e3c-e68963f38df5@192.168.204.9@tcp [ 3031.266295] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 3031.281557] Lustre: 101567:0:(ldlm_lib.c:1914:abort_req_replay_queue()) @@@ aborted: req@00000000963534a0 x1873480044844928/t0(8589934780) o35->321ee7c0-3964-49cd-9e3c-e68963f38df5@192.168.204.9@tcp:47/0 lens 392/0 e 0 to 0 dl 1786690937 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 3031.308065] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 3031.322882] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 3031.334946] Lustre: lustre-MDT0000: Denying connection for new client 321ee7c0-3964-49cd-9e3c-e68963f38df5 (at 192.168.204.9@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 3031.398064] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 3031.408118] Lustre: Skipped 2 previous similar messages [ 3031.430248] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 3031.497528] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 3031.500772] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:80 to 0x280000401:577) [ 3047.905148] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3047.909060] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3047.916209] LustreError: Skipped 1 previous similar message [ 3047.916900] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3047.916904] Lustre: Skipped 3 previous similar messages [ 3047.934236] Lustre: Skipped 4 previous similar messages [ 3060.191942] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3060.517555] Lustre: server umount lustre-OST0000 complete [ 3064.310688] LustreError: 101947:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3064.336328] LustreError: 101947:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 43 previous similar messages [ 3080.159177] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 3080.445576] Lustre: server umount lustre-OST0001 complete [ 3091.117451] Lustre: server umount lustre-MDT0000 complete [ 3094.938812] LustreError: 102162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786690990 with bad export cookie 8969651127657298665 [ 3094.943420] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3094.948390] LustreError: 102162:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3095.445795] Lustre: server umount lustre-MDT0001 complete [ 3105.862821] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 03:03:19 (1786690999) [ 3113.983639] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3113.991859] Lustre: *** cfs_fail_loc=197, val=0*** [ 3129.569244] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 3134.687528] LustreError: 103550:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.109@tcp: failed processing log, type 4: rc = -110 [ 3139.857365] Lustre: *** cfs_fail_loc=197, val=0*** [ 3139.864162] Lustre: Skipped 65 previous similar messages [ 3160.415881] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3160.424189] Lustre: Skipped 3 previous similar messages [ 3167.553772] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3176.412387] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3178.623399] Lustre: server umount lustre-OST0000 complete [ 3186.704911] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 03:04:40 (1786691080) [ 3209.199246] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3215.124735] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3225.674350] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3230.479322] LustreError: 105199:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 3230.495697] LustreError: 105199:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 3230.506158] LustreError: 105199:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 21, retries 0, failed: rc = -5 [ 3231.319388] Lustre: server umount lustre-MDT0000 complete [ 3271.258896] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 3284.296324] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 3324.894436] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3325.180717] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3325.211574] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3325.309651] Lustre: lustre-MDT0000: new disk, initializing [ 3325.430441] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3329.889848] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3342.638745] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3342.794927] Lustre: 109375:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3343.001482] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3348.180232] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3357.713089] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3363.222407] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3369.704066] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3369.992431] Lustre: 111343:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3375.691150] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3376.291723] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3385.363412] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3390.199814] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3390.404585] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3395.324893] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3395.556253] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3401.700816] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3401.710133] LustreError: Skipped 2 previous similar messages [ 3401.713909] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3401.742580] Lustre: Skipped 4 previous similar messages [ 3401.754717] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3401.765659] Lustre: Skipped 19 previous similar messages [ 3403.689090] Lustre: server umount lustre-OST0000 complete [ 3413.729610] Lustre: server umount lustre-MDT0000 complete [ 3417.570924] LustreError: 109384:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3417.605297] LustreError: 109384:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 3418.169584] LustreError: 109368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786691313 with bad export cookie 8969651127657309361 [ 3418.185509] LustreError: 109368:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 3418.192451] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3418.600949] Lustre: server umount lustre-MDT0001 complete [ 3429.106903] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 03:08:42 (1786691322) [ 3439.550857] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 3442.869272] Key type lgssc unregistered [ 3443.237743] LNet: 113836:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3443.243836] LNetError: 113836:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3443.270857] LNet: Removed LNI 192.168.204.109@tcp [ 3444.270886] Key type .llcrypt unregistered [ 3444.273635] Key type ._llcrypt unregistered [ 3463.586258] Key type ._llcrypt registered [ 3463.590834] Key type .llcrypt registered [ 3463.718314] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 3482.666472] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 3484.332036] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3484.420976] alg: No test for adler32 (adler32-zlib) [ 3485.436174] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 3485.653735] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 3487.279217] Key type lgssc registered [ 3489.100262] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3535.816719] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3535.853701] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3537.269730] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3537.351287] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3537.433359] Lustre: lustre-MDT0000: new disk, initializing [ 3537.516423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3537.551891] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3542.072672] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3554.896310] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3555.215957] Lustre: lustre-OST0000: new disk, initializing [ 3555.221956] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3555.227873] Lustre: Skipped 1 previous similar message [ 3555.234436] Lustre: 118012:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3555.371423] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3561.883288] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3563.060467] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 3563.098114] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 3563.136182] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 3573.451055] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3573.714962] Lustre: lustre-OST0001: new disk, initializing [ 3573.718788] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 3573.730857] Lustre: 118875:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3573.828629] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 3580.405542] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3583.085162] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 3583.095840] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 3583.128261] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 3593.188390] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3593.195052] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3593.209729] Lustre: Skipped 1 previous similar message [ 3593.224843] Lustre: Skipped 1 previous similar message [ 3596.181121] Lustre: server umount lustre-MDT0000 complete [ 3603.486712] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 3608.688684] LustreError: 117199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786691504 with bad export cookie 9171699897755084692 [ 3608.693466] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3608.698691] LustreError: 117199:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3608.890173] Lustre: server umount lustre-OST0000 complete [ 3613.285916] Lustre: server umount lustre-OST0001 complete [ 3624.373858] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 03:11:57 (1786691517) [ 3652.943284] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 03:12:26 (1786691546) [ 3664.023349] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 3691.666729] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 03:13:05 (1786691585) [ 3734.735357] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 3745.420930] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 3785.523400] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3785.832248] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 3785.891068] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 3785.998246] Lustre: lustre-MDT0000: new disk, initializing [ 3786.062744] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3786.085283] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 3790.651848] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3802.799912] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3802.905689] Lustre: 126640:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 3802.976242] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 3802.980871] Lustre: Skipped 1 previous similar message [ 3803.056978] Lustre: lustre-MDT0001: new disk, initializing [ 3803.253898] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3803.300192] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 3803.312531] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3808.562961] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3819.411834] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3825.725827] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3833.565419] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3833.808572] Lustre: lustre-OST0000: new disk, initializing [ 3833.811820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3833.817026] Lustre: 128611:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3833.890050] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3840.478295] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3841.622269] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3841.639408] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3841.753493] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3851.049713] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3857.513712] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 3857.865347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 3864.207828] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 3864.496836] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 3867.617996] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3867.625960] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 3867.640284] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3867.655750] Lustre: Skipped 1 previous similar message [ 3867.703585] LustreError: Skipped 1 previous similar message [ 3872.737654] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3872.743050] Lustre: Skipped 2 previous similar messages [ 3873.206965] Lustre: server umount lustre-OST0000 complete [ 3879.907172] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3879.915805] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3879.932110] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3882.978226] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3884.864270] Lustre: server umount lustre-MDT0000 complete [ 3889.098094] LustreError: 128610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786691784 with bad export cookie 9171699897755085917 [ 3889.101704] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3889.107947] LustreError: 128610:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 3889.739980] Lustre: server umount lustre-MDT0001 complete [ 3900.274458] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 03:16:33 (1786691793) [ 3907.967834] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3908.546684] LustreError: 131025:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3908.564556] LustreError: 131025:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 3908.641048] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3913.494443] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3913.703760] LustreError: 131026:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3918.820035] LustreError: 131025:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3922.840890] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3928.839257] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3937.019258] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3943.553566] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3951.177716] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3951.605702] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3951.615101] Lustre: Skipped 1 previous similar message [ 3959.607988] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 3968.975346] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3973.975048] Lustre: *** cfs_fail_loc=159, val=0*** [ 3973.979949] Lustre: Skipped 1 previous similar message [ 3990.379068] Lustre: lustre-MDT0000: Client 31540bf1-9890-418f-8d93-57040d37a020 (at 192.168.204.9@tcp) reconnecting [ 3992.251383] Lustre: *** cfs_fail_loc=159, val=0*** [ 3992.254430] Lustre: Skipped 5 previous similar messages [ 4007.788079] Lustre: lustre-MDT0000: Client 31540bf1-9890-418f-8d93-57040d37a020 (at 192.168.204.9@tcp) reconnecting [ 4014.049224] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4014.059763] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4014.069574] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4014.069581] Lustre: Skipped 2 previous similar messages [ 4017.755346] Lustre: server umount lustre-OST0000 complete [ 4024.288526] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4024.309192] Lustre: Skipped 1 previous similar message [ 4024.314600] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4024.327036] Lustre: Skipped 2 previous similar messages [ 4027.918338] Lustre: server umount lustre-MDT0000 complete [ 4029.409074] LustreError: 131992:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4029.424267] LustreError: 131992:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4032.193403] LustreError: 131005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786691927 with bad export cookie 9171699897755086799 [ 4032.196280] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4032.204593] LustreError: 131005:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 4032.907146] Lustre: server umount lustre-MDT0001 complete [ 4046.146062] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 4049.046674] Key type lgssc unregistered [ 4049.237042] LNet: 134803:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4049.249585] LNetError: 134803:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4049.276924] LNet: Removed LNI 192.168.204.109@tcp [ 4050.299564] Key type .llcrypt unregistered [ 4050.303546] Key type ._llcrypt unregistered [ 4076.544322] Key type ._llcrypt registered [ 4076.546579] Key type .llcrypt registered [ 4076.700874] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 4078.228207] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4078.287399] alg: No test for adler32 (adler32-zlib) [ 4079.553266] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 4079.910479] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 4081.607181] Key type lgssc registered [ 4082.857764] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4091.457630] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4091.491639] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4093.078707] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4098.552764] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4102.629079] LustreError: 136216:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4102.659119] LustreError: 136216:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4107.296637] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4107.696210] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4112.083214] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4120.598563] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4127.213780] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4133.573934] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4133.964463] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4138.046079] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 4141.362953] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4150.240898] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4163.175327] Lustre: *** cfs_fail_loc=159, val=0*** [ 4178.805525] Lustre: lustre-MDT0000: Client 1c31d968-c4e3-45dd-b547-b452528c276e (at 192.168.204.9@tcp) reconnecting [ 4182.205319] Lustre: *** cfs_fail_loc=159, val=0*** [ 4182.228327] Lustre: Skipped 1 previous similar message [ 4197.791953] Lustre: lustre-MDT0001: Client 1c31d968-c4e3-45dd-b547-b452528c276e (at 192.168.204.9@tcp) reconnecting [ 4199.916444] Lustre: *** cfs_fail_loc=159, val=0*** [ 4199.918368] Lustre: Skipped 2 previous similar messages [ 4216.187530] Lustre: lustre-MDT0001: Client 1c31d968-c4e3-45dd-b547-b452528c276e (at 192.168.204.9@tcp) reconnecting [ 4221.721335] Lustre: 139261:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4224.689325] Lustre: *** cfs_fail_loc=159, val=0*** [ 4224.691320] Lustre: Skipped 3 previous similar messages [ 4241.230895] Lustre: lustre-MDT0000: Client 07066c5e-52d6-4546-bed6-cfd7a9471e61 (at 192.168.204.9@tcp) reconnecting [ 4242.559501] Lustre: *** cfs_fail_loc=159, val=0*** [ 4242.566508] Lustre: Skipped 17 previous similar messages [ 4258.172839] Lustre: lustre-MDT0000: Client 07066c5e-52d6-4546-bed6-cfd7a9471e61 (at 192.168.204.9@tcp) reconnecting [ 4259.928750] Lustre: 139556:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4270.061057] LustreError: lustre-OST0000-osc-MDT0000: operation ost_destroy to node 0@lo failed: rc = -107 [ 4270.069844] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4270.094847] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4271.072365] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4271.077642] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4271.083505] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4271.114588] Lustre: Skipped 1 previous similar message [ 4272.536683] Lustre: server umount lustre-OST0000 complete [ 4276.704550] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4276.717707] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4276.734898] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4278.714991] Lustre: server umount lustre-MDT0000 complete [ 4281.313995] LustreError: 138975:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4281.354382] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4281.370453] LustreError: 138975:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4282.121663] LustreError: 139161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786692177 with bad export cookie 4140668729122070556 [ 4282.122985] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4282.130387] LustreError: 139161:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4282.457871] Lustre: server umount lustre-MDT0001 complete [ 4293.583263] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 4296.902569] Key type lgssc unregistered [ 4297.234681] LNet: 140617:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4297.251600] LNetError: 140617:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4297.269231] LNet: Removed LNI 192.168.204.109@tcp [ 4298.314877] Key type .llcrypt unregistered [ 4298.318553] Key type ._llcrypt unregistered [ 4326.603529] Key type ._llcrypt registered [ 4326.607663] Key type .llcrypt registered [ 4326.777167] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 4328.053526] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4328.086915] alg: No test for adler32 (adler32-zlib) [ 4329.251683] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 4329.502703] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 4331.183157] Key type lgssc registered [ 4332.521813] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4341.379942] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4341.418459] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4343.228778] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4348.121353] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4352.489378] LustreError: 142031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4352.508730] LustreError: 142031:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4357.605148] LustreError: 142031:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4358.515215] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4359.194175] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4364.690807] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4374.203725] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4381.407524] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4389.616366] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4390.061830] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4397.791894] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4403.404894] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 4407.102140] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4413.668637] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 4418.963501] Lustre: 144616:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4423.872386] Lustre: 144665:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4426.068526] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 4427.399577] Lustre: Setting parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=8 in log params [ 4434.161680] Lustre: 144822:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 4435.259343] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 4436.577656] Lustre: 144921:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 4444.641466] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4444.653788] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4444.670288] LustreError: Skipped 1 previous similar message [ 4444.671774] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4444.689120] Lustre: Skipped 1 previous similar message [ 4444.707845] Lustre: Skipped 1 previous similar message [ 4448.532692] Lustre: server umount lustre-OST0000 complete [ 4454.888651] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4454.900886] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4454.909309] Lustre: Skipped 1 previous similar message [ 4454.927213] Lustre: Skipped 1 previous similar message [ 4460.001782] LustreError: 142976:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4460.016959] LustreError: 142976:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4460.195415] Lustre: server umount lustre-MDT0000 complete [ 4465.135018] LustreError: 145029:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4465.177433] LustreError: 145029:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4466.149781] LustreError: 142013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786692361 with bad export cookie 7817050603165862713 [ 4466.150870] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4466.176930] LustreError: 142013:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4466.928501] Lustre: server umount lustre-MDT0001 complete [ 4480.977709] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 4484.283754] Key type lgssc unregistered [ 4484.538578] LNet: 145986:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4484.560357] LNetError: 145986:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4485.607491] LNet: Removed LNI 192.168.204.109@tcp [ 4486.572989] Key type .llcrypt unregistered [ 4486.575447] Key type ._llcrypt unregistered [ 4512.189667] Key type ._llcrypt registered [ 4512.195402] Key type .llcrypt registered [ 4512.347099] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 4513.479618] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4513.558267] alg: No test for adler32 (adler32-zlib) [ 4514.624779] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 4514.819747] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 4516.487988] Key type lgssc registered [ 4517.855375] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4526.345977] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4526.372928] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4527.976327] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4532.574656] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4538.342639] LustreError: 147419:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4538.390994] LustreError: 147419:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4542.500186] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4542.928391] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4548.821920] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4556.766715] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4562.596462] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4570.554741] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4570.989673] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4573.051642] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 4577.485347] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4585.090949] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4587.408632] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 4592.017563] Lustre: *** cfs_fail_loc=159, val=0*** [ 4608.367919] Lustre: lustre-MDT0000: Client 82528f8a-409c-40f6-9e80-e03dc1ecd3ae (at 192.168.204.9@tcp) reconnecting [ 4618.214832] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4618.220713] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4618.236529] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4618.284813] Lustre: Skipped 1 previous similar message [ 4620.835585] Lustre: server umount lustre-OST0000 complete [ 4628.467211] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4628.477712] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4628.483437] Lustre: Skipped 1 previous similar message [ 4628.504914] Lustre: Skipped 2 previous similar messages [ 4633.572219] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4633.574995] Lustre: Skipped 1 previous similar message [ 4637.685756] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4637.696246] Lustre: Skipped 1 previous similar message [ 4640.223449] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 4640.618674] Lustre: server umount lustre-MDT0000 complete [ 4642.788797] LustreError: 147420:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4642.812660] LustreError: 147420:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4645.567183] LustreError: 147399:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786692541 with bad export cookie 10513743513604627740 [ 4645.585026] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4646.052514] Lustre: server umount lustre-MDT0001 complete [ 4658.969250] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 4662.883425] Key type lgssc unregistered [ 4663.197739] LNet: 151090:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4663.210577] LNetError: 151090:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4663.242607] LNet: Removed LNI 192.168.204.109@tcp [ 4664.527333] Key type .llcrypt unregistered [ 4664.529061] Key type ._llcrypt unregistered [ 4696.142784] Key type ._llcrypt registered [ 4696.144896] Key type .llcrypt registered [ 4696.296096] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 4697.312174] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4697.325084] alg: No test for adler32 (adler32-zlib) [ 4698.756694] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 4699.093281] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 4700.848531] Key type lgssc registered [ 4702.580988] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4715.347982] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4715.364390] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4717.002423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4721.936581] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4727.275709] LustreError: 152506:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4727.286824] LustreError: 152506:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 4731.886445] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4732.520429] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4738.438609] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4747.316679] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4754.365965] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4762.677561] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4763.131496] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4770.806459] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 4777.463082] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 4779.086917] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4786.681487] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 4811.017220] Lustre: lustre-MDT0001: evicting 8336f71c-4da7-4df0-9d58-2de9d93d625c (at 192.168.204.9@tcp) by administrative request [ 4811.031356] Lustre: Skipped 1 previous similar message [ 4842.980738] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4842.994562] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 4842.995092] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4843.014579] Lustre: Skipped 1 previous similar message [ 4847.675700] Lustre: server umount lustre-OST0000 complete [ 4853.217688] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4853.234441] Lustre: Skipped 1 previous similar message [ 4853.243933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4858.343721] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4858.359978] Lustre: Skipped 2 previous similar messages [ 4859.278820] Lustre: server umount lustre-MDT0000 complete [ 4862.437702] LustreError: 152500:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4862.472915] LustreError: 152500:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4864.051686] LustreError: 152485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786692759 with bad export cookie 14359801497125439320 [ 4864.055427] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4864.070859] LustreError: 152485:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 4864.643886] Lustre: server umount lustre-MDT0001 complete [ 4879.145625] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 4882.613630] Key type lgssc unregistered [ 4882.982839] LNet: 156605:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4883.003714] LNetError: 156605:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4883.028149] LNet: Removed LNI 192.168.204.109@tcp [ 4884.391218] Key type .llcrypt unregistered [ 4884.395533] Key type ._llcrypt unregistered [ 4977.921714] Key type ._llcrypt registered [ 4977.923368] Key type .llcrypt registered [ 4978.162758] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 4979.407915] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4979.427562] alg: No test for adler32 (adler32-zlib) [ 4980.786197] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 4981.038057] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 4982.831374] Key type lgssc registered [ 4984.263900] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4992.896598] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4992.912664] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4994.489356] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4999.700955] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5004.772466] LustreError: 159098:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5004.793626] LustreError: 159098:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5009.889667] LustreError: 159099:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5009.972286] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5010.396863] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5016.399036] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5026.045704] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5033.168436] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5043.032437] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5043.739541] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5051.844341] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5054.961472] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 5062.051985] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5065.231337] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 5080.551458] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 5080.558039] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5080.566451] LustreError: Skipped 1 previous similar message [ 5080.587748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5080.607719] Lustre: Skipped 1 previous similar message [ 5081.755534] Lustre: server umount lustre-OST0000 complete [ 5087.202931] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5087.216812] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5087.246713] Lustre: Skipped 1 previous similar message [ 5087.259635] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5090.786083] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5090.786344] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5092.542791] Lustre: server umount lustre-MDT0000 complete [ 5095.905033] LustreError: 160348:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5095.952390] LustreError: 160348:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5097.841634] LustreError: 159080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786692993 with bad export cookie 1157767989994163561 [ 5097.846873] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5097.887261] LustreError: 159080:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5098.864464] Lustre: server umount lustre-MDT0001 complete [ 5112.147402] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 5115.135808] Key type lgssc unregistered [ 5115.526672] LNet: 162668:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5115.535547] LNetError: 162668:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5115.561250] LNet: Removed LNI 192.168.204.109@tcp [ 5116.893763] Key type .llcrypt unregistered [ 5116.895036] Key type ._llcrypt unregistered [ 5153.021327] Key type ._llcrypt registered [ 5153.024113] Key type .llcrypt registered [ 5153.157243] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 5154.105688] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5154.224135] alg: No test for adler32 (adler32-zlib) [ 5155.240430] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 5155.611092] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 5157.303131] Key type lgssc registered [ 5158.635799] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5178.289519] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 03:37:51 (1786693071) [ 5198.666878] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 5211.463848] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 5258.690478] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5258.724861] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5260.059769] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5260.120274] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5260.235452] Lustre: lustre-MDT0000: new disk, initializing [ 5260.377934] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5260.442777] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5266.459523] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5282.120524] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5282.252173] Lustre: 166940:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5282.346051] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5282.350632] Lustre: Skipped 1 previous similar message [ 5282.457061] Lustre: lustre-MDT0001: new disk, initializing [ 5282.576776] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5282.607609] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5282.621725] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5288.813837] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5299.599867] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5305.987500] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5315.309764] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5315.759394] Lustre: lustre-OST0000: new disk, initializing [ 5315.763370] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5315.770331] Lustre: 168909:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5315.860847] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5323.881097] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5323.894522] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5324.061199] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5324.304950] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5335.638305] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5349.857534] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5349.866419] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5349.877229] LustreError: Skipped 1 previous similar message [ 5349.879749] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5349.943342] Lustre: Skipped 1 previous similar message [ 5354.076410] Lustre: server umount lustre-OST0000 complete [ 5359.594408] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5359.623417] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5359.651163] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5359.656822] Lustre: Skipped 1 previous similar message [ 5365.011261] Lustre: server umount lustre-MDT0000 complete [ 5369.576798] LustreError: 166933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786693265 with bad export cookie 6317551111988885501 [ 5369.589681] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5369.593641] LustreError: 166933:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 5370.133927] Lustre: server umount lustre-MDT0001 complete [ 5383.920396] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 5387.217253] Key type lgssc unregistered [ 5387.561770] LNet: 170819:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5387.580888] LNetError: 170819:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5387.607770] LNet: Removed LNI 192.168.204.109@tcp [ 5388.674125] Key type .llcrypt unregistered [ 5388.676628] Key type ._llcrypt unregistered [ 5421.901465] Key type ._llcrypt registered [ 5421.907528] Key type .llcrypt registered [ 5422.082863] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 5424.378099] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5424.445180] alg: No test for adler32 (adler32-zlib) [ 5425.937502] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 5426.533920] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 5428.529587] Key type lgssc registered [ 5430.625582] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5441.007617] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5441.049042] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5442.733553] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5448.676653] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5453.284287] LustreError: 172237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5453.322467] LustreError: 172237:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5458.414719] LustreError: 172237:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5458.934590] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5459.526266] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5464.477534] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5474.936223] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5481.294989] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5489.250667] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5489.887086] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5497.908110] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 5507.000899] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5512.455647] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 5518.248988] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5518.262024] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5518.283218] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5518.288362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5518.306376] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5519.436978] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5519.462972] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5519.478979] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5519.490320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5519.504084] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5520.575712] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5520.590356] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5520.602073] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5520.606448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5520.616638] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5521.481043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5522.466589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.405766] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5523.414323] Lustre: Skipped 2 previous similar messages [ 5523.443254] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5523.461210] Lustre: Skipped 2 previous similar messages [ 5523.467641] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5523.475772] Lustre: Skipped 2 previous similar messages [ 5523.482578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5523.500380] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5523.512726] Lustre: Skipped 2 previous similar messages [ 5524.531266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5525.838222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5526.748408] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.609443] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5527.613979] Lustre: Skipped 3 previous similar messages [ 5527.626030] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5527.635698] Lustre: Skipped 3 previous similar messages [ 5527.639729] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5527.645656] Lustre: Skipped 3 previous similar messages [ 5527.649847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5527.659906] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5527.668631] Lustre: Skipped 3 previous similar messages [ 5528.759971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5529.801788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5530.779237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5531.678658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5532.748692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5534.076060] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5535.210994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.431383] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5536.443689] Lustre: Skipped 7 previous similar messages [ 5536.475762] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5536.491676] Lustre: Skipped 7 previous similar messages [ 5536.500431] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5536.506851] Lustre: Skipped 7 previous similar messages [ 5536.531289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5536.560295] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5536.578654] Lustre: Skipped 7 previous similar messages [ 5537.909399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5539.215275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5540.675451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5541.921382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5542.994823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5544.164432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5545.458380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5546.684757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5547.754623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5548.854382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5549.729243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5551.040882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5552.413672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.445878] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5553.454422] Lustre: Skipped 13 previous similar messages [ 5553.472116] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5553.482224] Lustre: Skipped 13 previous similar messages [ 5553.485764] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5553.496060] Lustre: Skipped 13 previous similar messages [ 5553.499861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5553.509616] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5553.520078] Lustre: Skipped 13 previous similar messages [ 5554.656676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5555.744612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5556.646343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5557.551672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5558.744643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5559.830740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5560.703431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5561.708702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5562.564382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5563.431810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5564.491632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5565.515976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5566.457802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5567.399487] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.183511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5568.999373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5570.032088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5571.050974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5572.383702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5573.422732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5574.646139] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5575.522793] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5576.536763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5577.436437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5578.156238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5579.361659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5580.307899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5581.484214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5582.453721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5583.553492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5584.588685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.868086] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5585.875508] Lustre: Skipped 31 previous similar messages [ 5585.901732] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5585.910812] Lustre: Skipped 31 previous similar messages [ 5585.914900] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5585.923531] Lustre: Skipped 31 previous similar messages [ 5585.926895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5585.941304] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5585.946513] Lustre: Skipped 31 previous similar messages [ 5586.870752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5587.934724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5588.881215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5589.954575] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5590.930717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5591.854844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5593.703401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5595.678450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5596.878504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5598.332403] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5599.832837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5601.707242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5603.450478] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5604.792728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5605.810578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5607.657238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5609.494839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5610.852108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5613.022156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5614.555732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5615.994467] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5617.076520] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5618.245288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5619.399679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5620.713892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5621.833699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5623.341147] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5624.832307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5626.243532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5627.152782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5628.253261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5629.198960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5630.485462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5631.881987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5632.887930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5634.518674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5636.050872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5637.411481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5638.597489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5639.627857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5640.458785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5641.507614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5642.481160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5643.442498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5644.381748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5645.275399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5646.142337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5647.140563] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5648.225394] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5649.650782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.049991] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5651.062234] Lustre: Skipped 50 previous similar messages [ 5651.082215] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5651.092225] Lustre: Skipped 50 previous similar messages [ 5651.095784] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5651.100857] Lustre: Skipped 50 previous similar messages [ 5651.105109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5651.118455] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5651.125523] Lustre: Skipped 50 previous similar messages [ 5652.427274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5654.100271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5655.312975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5656.746314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5657.988919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5659.449376] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5661.031156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5662.313260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5663.591199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5664.834588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5665.936705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5666.972907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5667.962229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5668.718435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5669.826732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5671.188448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5672.413089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5673.251314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5674.191752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5675.209092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5676.365883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5677.325765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5678.480453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5679.312622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5680.343204] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5681.418955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5682.483975] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5683.343281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5684.253654] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5685.067971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5686.040400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.001047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5687.973557] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5688.893782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5689.907738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.036475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5691.934661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5692.771603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5693.603347] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5694.632351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5695.536446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5696.643771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5697.760632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5698.951609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5700.169397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5701.316149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5702.321493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5703.288739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5704.266110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5705.411414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5706.741465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5707.924827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5708.948217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5710.007080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5711.096368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5712.404319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5713.636921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5714.464183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5715.622953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5716.992188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5718.064240] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5719.348196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5720.592681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5721.756160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5722.862289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5724.068788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5725.066274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5726.258094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5727.268531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5728.405859] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5729.524899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5730.641448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5731.730311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5732.800071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5733.985886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5735.027678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.122764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5736.991229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5737.810733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5739.088993] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5740.362319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5741.659614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5742.757838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5744.253474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5745.341751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5746.453531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5747.807980] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5748.999573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5750.233062] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5751.046692] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5752.038286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5753.127166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5754.391275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5755.207511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5756.317261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5757.255197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5758.700918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5760.058906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5761.056109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5762.224766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5763.612677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5764.702756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5765.668704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5766.795065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5767.836625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5768.722454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5769.923206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5771.055591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5772.344757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5773.470154] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5774.402561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5775.406535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5776.349659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5777.404243] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5778.545739] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.389715] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 5779.393221] Lustre: Skipped 115 previous similar messages [ 5779.404301] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5779.413245] Lustre: Skipped 115 previous similar messages [ 5779.417084] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 5779.425667] Lustre: Skipped 115 previous similar messages [ 5779.438341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5779.453769] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 5779.465955] Lustre: Skipped 115 previous similar messages [ 5780.493509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5781.911358] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5782.783521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5783.624276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5784.493946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5785.880883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5787.220270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5788.398603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5789.319293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5790.942327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5792.092126] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5793.173163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5794.557741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5795.686823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5796.905617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5798.158239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5799.321482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5800.340274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5801.562882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5802.562989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5803.671760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5804.716522] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5805.763903] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5806.477378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5807.298569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5808.015185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5809.002915] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.022332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5810.836662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5811.731434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5812.748655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5813.732989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5814.683373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5815.808786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5816.727073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5817.698532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5818.509719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5819.237361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5820.264962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5821.233996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.075313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5822.896559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5823.787966] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5824.858577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5825.857288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5826.958545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5827.947328] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5829.026845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5830.155176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5831.070911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5832.055390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5833.205710] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5834.176933] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5835.138232] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5836.068528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5837.172954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5838.221341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5839.222896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5840.239580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5841.290112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5842.377473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5843.453551] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5844.639224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5845.527684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5846.523761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5847.697346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5848.722605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5849.755222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5850.809935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5851.738265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5852.733495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5853.638247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5854.504146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5855.415856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5856.468277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5857.494713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5858.595345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5859.790516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5860.913316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5862.114784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5863.082882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5864.213208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5865.320861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5866.269344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5867.288781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5868.353183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5869.493508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5870.586623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5871.578277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5872.667706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5873.686606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5874.879363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5876.087679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5877.130366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5878.041317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5879.138727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5880.337738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5881.565630] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5882.603462] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5883.609057] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5884.524161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5885.439499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5886.348526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5887.219313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5888.259064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5889.334969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5890.210477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.135380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5891.993228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5892.977492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5893.758564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5894.683221] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5895.641146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5896.569251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5897.533120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5898.552882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5899.433835] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5900.421703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5901.314443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5902.241785] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5903.188034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.149451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5904.990886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5905.925392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5906.941579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5907.794237] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5908.597521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5909.440860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5910.393049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5911.347483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5912.281965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5913.211500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5914.308273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5915.228286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.138677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5916.957188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5917.767168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5918.596495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5919.502853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5920.462513] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5921.494807] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5922.371955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5923.325659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5924.230801] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.163705] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5925.976750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5926.882591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5927.752508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5928.778760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5929.775857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5930.736461] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5931.565377] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5932.665677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5933.642697] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5934.682533] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5935.822836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5936.885206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5937.934278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5938.885891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5939.922936] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5940.866871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5941.822819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5942.682066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5943.703663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5944.657814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5945.798820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5946.829077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5947.720976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5948.626878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5949.593206] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5950.455728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5951.469413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5952.413847] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5953.268354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.087229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5954.998064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5955.981798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.075525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5957.950209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5959.265620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5960.469938] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5961.495482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5962.817876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5963.810662] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5965.373727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5966.936958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5967.766184] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5968.874158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5969.741579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5970.928070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5972.098779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5973.115735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5974.323201] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5975.463916] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5976.592955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5977.899837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5978.983177] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.051599] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5980.958837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5981.934901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5982.677367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5983.684846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5984.571439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5985.627409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5986.597726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5987.705061] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5988.828773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5989.961076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5990.899976] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5991.963270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5993.018192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5994.362911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5995.449950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5996.385409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5997.534214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5998.430752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 5999.371782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6000.138946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6001.208711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6002.176381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6003.148171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.134939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6004.902484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6005.796083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6006.768443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6007.836607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6008.899282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6009.906625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6010.904843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6012.137300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6013.070479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6014.050704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6015.118293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6016.174678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6017.199374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6018.361306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6019.307510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6020.370609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6021.265531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6022.391512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6023.580867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6024.630792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6025.667992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6026.913161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6028.008161] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6029.251539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6030.329688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6031.346749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6032.363624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6033.404372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6034.539415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.835360] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6035.844433] Lustre: Skipped 251 previous similar messages [ 6035.864451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6035.890453] Lustre: Skipped 251 previous similar messages [ 6035.897622] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6035.910414] Lustre: Skipped 251 previous similar messages [ 6035.919667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6035.930587] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6035.933101] Lustre: Skipped 251 previous similar messages [ 6037.000676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6038.055497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6039.150265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6040.112511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.110687] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6041.992365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.008082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6043.950074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6044.917334] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6045.637152] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6046.467119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6047.338175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6048.230422] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6049.124355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.034845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6050.979828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6051.992684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6052.814810] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6053.674157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6054.575407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6055.770451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6056.541948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6057.503672] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6058.481073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6059.458791] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6060.343972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6061.365524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6062.404928] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6063.266935] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6064.180230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6065.128432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6066.035135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6067.012658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6068.068608] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6069.099531] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6070.102823] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6071.065490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6072.144133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6073.136270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6074.163803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6075.194288] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6076.147174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6077.051910] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6078.021695] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6078.030744] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6078.046170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6079.130888] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6080.241562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6081.314460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6082.457346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6083.617382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6084.600503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6085.622482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6086.550287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6087.630263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6088.588733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6089.662558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6090.676784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6091.932294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6093.162863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6094.213789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6095.324090] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6096.567086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6097.507397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6098.786321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6099.733404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6100.953418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6101.842320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6102.901064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6103.902508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6104.998160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6105.915585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6106.832140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6108.051137] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6109.262722] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6109.279087] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6109.303277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6110.587897] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6111.587277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6112.685837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6113.826180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6114.825287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6115.745330] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6116.757128] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6117.834317] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6118.796407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6119.808902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6120.829839] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6121.759396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6122.769668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6123.979989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6124.928777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6125.920217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6126.959991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6128.104375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6129.047228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6130.015826] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6131.068694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6132.113477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6133.091319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6134.018357] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.008173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6135.891255] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6136.733482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6137.710833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6138.658570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6139.751064] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6140.636365] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6141.676624] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6142.585953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6143.611770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6144.540671] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6145.705899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6146.684875] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6147.609231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6148.516598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6149.506778] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6150.481841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6151.412203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6152.334707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6153.335150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6154.254865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6155.343802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6156.251388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6157.314900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6158.232124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6159.167121] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6160.086965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6161.093289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6162.214440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6163.258214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6164.253199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6165.283259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6166.484367] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6167.542816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6168.664490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6169.783894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6171.146727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6172.575112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6173.844758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6175.280173] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6176.833493] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6178.081158] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6179.268568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.127481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6180.987902] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6182.151335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6183.318695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6184.425238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6185.680760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6186.755089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6187.767475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6189.031044] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6190.351857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6191.229411] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6192.161631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6193.302749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6194.320374] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6195.386448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6196.336879] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6197.378525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6198.628656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6199.909969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6200.852448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6201.985314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6203.020352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6204.093460] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6205.012625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6206.016597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6207.147373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6208.149172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6209.145881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6210.183216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6211.140874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6212.158556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6213.155720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6214.033939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6215.032977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6216.021657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6217.013673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6218.114855] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6219.255501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6220.404923] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6220.427000] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6220.443637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6221.700079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6222.632413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6223.545917] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6224.362799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6225.197703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6226.297733] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6227.502333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6228.846426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6230.188496] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6231.386313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6232.399706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6233.257626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6234.245838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6235.599906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6236.702401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6237.778709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6238.764595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6239.966174] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6241.128707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6242.417375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6243.612717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6244.814740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6245.955089] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6247.460595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6249.123693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6250.269241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6251.348203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6252.726241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6254.011293] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6254.026351] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6254.055271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6255.263701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6256.417195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6257.577340] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6258.916415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6260.287770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6261.665614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6262.847103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6264.028323] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6265.179458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6266.429291] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6267.337868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6268.480131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6269.783670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6270.674512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6271.764431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6272.893636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6273.984579] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6275.197704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6276.286770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6277.535075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6278.645824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6279.671548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6280.770307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6281.706872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6282.708244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6283.776399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6284.790180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6285.691773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6286.622474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6287.595734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6288.609920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6289.874175] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6291.204783] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6292.415564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6293.459828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6294.537770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6295.580656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6296.607375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6297.547721] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6298.643069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6299.789536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6300.798454] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6301.889171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6302.814313] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6303.873268] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6304.890732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6305.877952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6306.993942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6307.994805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6309.320066] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6310.516322] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6311.793806] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6312.927166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6313.956051] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6315.077339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6316.452735] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6317.761530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6319.060351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6320.474602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6321.813072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6322.773450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6324.045077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6325.391843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6326.695264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6327.606937] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6328.788890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6329.833192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6331.037032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6332.388983] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6333.756703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6334.880495] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6335.839226] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6336.988446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6337.943499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6339.074617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6340.151535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6341.402345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6342.665929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6343.934878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6345.224713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6346.497106] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6347.755047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6349.163231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6350.351459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6351.374512] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6352.797157] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6354.088849] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6355.253229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6356.313228] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6357.535684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6358.726190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6359.881886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6360.833745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6362.208341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6363.452887] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6364.559814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6365.784952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6367.012369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6368.056443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6369.157443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6370.116360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.064948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6371.954043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6372.997634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6374.065679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6375.173600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6376.168223] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6377.189699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6378.122740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6379.354800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6380.361294] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6381.763995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6383.142093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6384.283169] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6385.304583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6386.283054] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6387.475114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6388.505755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6389.557548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6390.815171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6391.828109] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6392.775561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6393.797573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6394.979404] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6395.982501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6397.142870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6398.106046] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6399.291312] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6400.430814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6401.645682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6402.676947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6403.650804] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6404.677356] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6405.804504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6406.735500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6407.969042] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6408.987532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6409.982568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6411.118203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6412.337361] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6413.274118] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6414.325197] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6415.406892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6416.518856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6417.608874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6418.658939] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6420.032289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6421.060080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6422.255218] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6423.077491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6424.223560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6425.102123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6426.191292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6427.463063] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6428.722725] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6429.942293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6431.057244] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6432.055972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6433.111610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6434.247202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6435.319952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6436.352734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6437.491501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6438.606982] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6439.910880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6441.004958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6442.027172] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6443.130730] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6444.135996] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6445.113958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6446.270553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6447.371530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6448.615947] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6449.680050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6450.741695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6451.847338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6452.895669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6454.219435] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6455.369874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6456.669390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6457.788695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6458.726179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6459.833381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6460.846363] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6462.057069] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6463.327821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6464.514070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6465.873535] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6466.965732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6468.153498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6469.264514] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6470.325560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6471.270994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6472.376924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6473.594747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6474.862390] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6475.982756] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6477.107485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6478.417925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6479.527709] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6480.653547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6481.918728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6483.154224] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6484.405516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6485.654086] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6486.577130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6487.585584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6489.218473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6490.860718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6492.501676] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6493.655870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6494.730441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6496.185925] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6497.655930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6498.832836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6499.963368] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6499.995184] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6500.005890] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6501.137185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6502.559536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6503.574831] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6504.849336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6506.319233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6507.648901] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6508.991705] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6509.016172] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6509.019617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6510.032623] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6511.370770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6512.652481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6514.347094] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6515.931401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6517.216486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6518.716185] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6520.432423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6521.558259] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6523.026748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6524.515192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6525.641864] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6527.213815] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6528.389809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6529.838911] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6531.043775] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6532.242882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6533.631805] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6535.257425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6536.503601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6537.786486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6538.962611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6539.791812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6540.887728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6541.973895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6543.124870] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6544.423635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6545.550419] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6546.809970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.279419] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 6548.291573] Lustre: Skipped 460 previous similar messages [ 6548.305482] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6548.323287] Lustre: Skipped 460 previous similar messages [ 6548.334782] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 6548.344469] Lustre: Skipped 460 previous similar messages [ 6548.351188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6548.385968] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 6548.402491] Lustre: Skipped 460 previous similar messages [ 6549.848850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6551.329418] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6552.744236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6553.771820] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6555.051426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6556.183813] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6557.493610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6558.613065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6559.716366] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6560.898617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6561.832819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6563.002972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6564.259648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6565.390246] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6566.690296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6568.009601] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6569.328108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6570.568530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6571.742562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6572.976559] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6574.319639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6575.537816] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6576.718851] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6577.758811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6578.904878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6580.217796] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6581.207266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6582.177485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6583.125803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6584.565028] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6585.764142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6587.085135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6588.247515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6589.507738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6590.783087] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6592.031038] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6593.456940] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6594.703799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6595.703695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6596.738215] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 6596.750105] LustreError: 172257:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 6596.765764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6597.966372] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6599.090574] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6600.343657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6601.591483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6602.921945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6604.249146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6605.711561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6606.992117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6608.203538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6609.283737] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6610.484194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6611.578099] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6612.647463] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6613.831146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6614.870688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 6615.920841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 7177.953900] LustreError: 226112:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 7177.965860] LustreError: 172257:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 7942.628671] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7942.638335] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7942.648139] Lustre: Skipped 56 previous similar messages [ 7942.656459] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7944.673580] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7944.684467] Lustre: Skipped 1 previous similar message [ 7948.271318] Lustre: server umount lustre-OST0000 complete [ 7952.864555] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7952.871087] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7954.916086] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7954.925457] Lustre: Skipped 1 previous similar message [ 7958.876682] Lustre: server umount lustre-MDT0000 complete [ 7960.036649] LustreError: 172238:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7960.050835] LustreError: 172238:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7962.484151] LustreError: 172217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786695858 with bad export cookie 8216416582040345392 [ 7962.501782] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7962.504624] LustreError: 172217:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7962.921550] Lustre: server umount lustre-MDT0001 complete [ 7975.098898] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 7977.993517] Key type lgssc unregistered [ 7978.394937] LNet: 227572:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7978.401835] LNetError: 227572:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7978.431758] LNet: Removed LNI 192.168.204.109@tcp [ 7979.318315] Key type .llcrypt unregistered [ 7979.320647] Key type ._llcrypt unregistered [ 8008.424912] Key type ._llcrypt registered [ 8008.429952] Key type .llcrypt registered [ 8008.718447] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8010.427857] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8010.458091] alg: No test for adler32 (adler32-zlib) [ 8011.679664] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 8012.295878] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 8014.081370] Key type lgssc registered [ 8015.733522] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8025.640602] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8025.669846] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8027.259452] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8032.606665] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8036.849184] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8036.864612] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8041.953734] LustreError: 228991:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8042.754562] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8043.242076] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8047.877869] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8056.786369] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8063.474341] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8070.815768] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8071.263189] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8072.317437] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 8078.084741] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8086.769908] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8092.155383] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 8094.117937] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8094.139245] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 8094.150200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8094.165432] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8094.165469] LustreError: 229010:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 8098.277769] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8098.282939] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8098.285149] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8098.307562] Lustre: Skipped 1 previous similar message [ 8099.152855] Lustre: lustre-OST0000: Not available for connect from 192.168.204.9@tcp (stopping) [ 8099.161707] Lustre: Skipped 1 previous similar message [ 8102.700791] Lustre: server umount lustre-OST0000 complete [ 8114.167114] LustreError: 229010:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 8137.308353] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8137.676316] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8138.736581] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8138.766588] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 8138.777447] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 8138.794644] Lustre: Skipped 1 previous similar message [ 8145.015545] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8153.373650] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8155.125749] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 8160.682295] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0a891d08000.ost_server_uuid 50 [ 8162.212792] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0a891d08000.ost_server_uuid in IDLE state after 0 sec [ 8167.398537] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 8167.409757] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8167.433220] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8169.446173] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 8169.456411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 8169.462384] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8169.491877] Lustre: Skipped 1 previous similar message [ 8173.569976] Lustre: server umount lustre-OST0000 complete [ 8179.684757] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8179.688988] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8179.693425] Lustre: Skipped 1 previous similar message [ 8184.092303] Lustre: server umount lustre-MDT0000 complete [ 8184.804283] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8184.829091] LustreError: 228990:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8188.733551] LustreError: 228970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696084 with bad export cookie 18189548485630479727 [ 8188.736709] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8188.747381] LustreError: 228970:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8189.170052] Lustre: server umount lustre-MDT0001 complete [ 8201.374822] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 8205.089820] Key type lgssc unregistered [ 8205.366894] LNet: 234201:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8205.378143] LNetError: 234201:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8205.401930] LNet: Removed LNI 192.168.204.109@tcp [ 8206.313719] Key type .llcrypt unregistered [ 8206.321044] Key type ._llcrypt unregistered [ 8241.296838] Key type ._llcrypt registered [ 8241.299176] Key type .llcrypt registered [ 8241.387465] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8242.818862] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 8242.998461] alg: No test for adler32 (adler32-zlib) [ 8244.060168] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 8244.296946] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 8245.975380] Key type lgssc registered [ 8247.175040] Lustre: Echo OBD driver; http://www.lustre.org/ [ 8257.381375] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 8257.404146] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8258.967803] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8264.626484] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8269.285687] LustreError: 235933:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8269.306771] LustreError: 235933:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 8273.295698] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8273.639983] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8278.215197] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8281.542983] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 8288.722772] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8289.161131] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8296.056691] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8303.540909] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 8305.282044] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8305.507834] Lustre: lustre-OST0001: new disk, initializing [ 8305.514644] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8305.524109] Lustre: 238242:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8305.662198] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 8312.633324] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8313.514191] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8313.525597] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8313.649402] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 8323.484324] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8329.897712] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 8345.577223] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8345.586603] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8345.606465] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8349.677517] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8349.680941] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8349.693916] Lustre: Skipped 1 previous similar message [ 8349.722644] Lustre: Skipped 3 previous similar messages [ 8351.674797] Lustre: server umount lustre-MDT0000 complete [ 8354.797136] LustreError: 235928:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8354.836638] LustreError: 235928:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8356.545680] LustreError: 235914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696252 with bad export cookie 4186175391495347242 [ 8356.552919] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8356.561313] LustreError: 235914:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8357.187894] Lustre: server umount lustre-MDT0001 complete [ 8371.469862] Lustre: server umount lustre-OST0000 complete [ 8386.499172] Lustre: server umount lustre-OST0001 complete [ 8411.837511] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8422.810330] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8423.546362] LustreError: 241350:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8423.653290] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8428.274267] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8429.033236] LustreError: 241351:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8434.155795] LustreError: 241350:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8437.906978] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8438.363931] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8443.639257] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8446.908932] Lustre: 242493:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8454.904164] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8455.379212] Lustre: mylustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8462.591859] LustreError: 242845:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8463.401108] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8469.085297] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49760) [ 8473.496337] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8480.953335] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8489.257512] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8493.390456] Lustre: 244364:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8504.801843] LustreError: mylustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8504.820891] Lustre: mylustre-MDT0000-osp-MDT0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8504.839533] Lustre: Skipped 1 previous similar message [ 8504.852857] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8509.411981] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8509.414617] Lustre: mylustre-MDT0000-lwp-OST0001: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8509.429445] Lustre: Skipped 2 previous similar messages [ 8509.457018] Lustre: Skipped 2 previous similar messages [ 8510.866521] Lustre: server umount mylustre-MDT0000 complete [ 8514.534658] LustreError: 243156:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8514.545313] LustreError: 243156:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 8515.300229] LustreError: 241326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696410 with bad export cookie 4186175391495350049 [ 8515.302925] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8515.314467] LustreError: 241326:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8515.713356] Lustre: server umount mylustre-MDT0001 complete [ 8530.172707] Lustre: server umount mylustre-OST0000 complete [ 8545.339189] Lustre: server umount mylustre-OST0001 complete [ 8571.227134] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8583.031944] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8583.515862] LustreError: 246678:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8583.679739] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8583.694272] Lustre: Skipped 1 previous similar message [ 8588.454306] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8599.122278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8605.404664] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8608.911472] Lustre: 247823:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8616.429312] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8623.471183] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8629.199428] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49792) [ 8632.960834] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8638.475648] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:65) [ 8641.394314] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8649.116740] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8652.854068] Lustre: 249695:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8660.963087] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8660.979170] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8661.001363] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 8661.013869] Lustre: Skipped 1 previous similar message [ 8662.499903] Lustre: server umount tfs-MDT0000 complete [ 8664.033963] LustreError: 247628:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8664.069680] LustreError: 247628:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 8666.880970] LustreError: 247825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696562 with bad export cookie 4186175391495351715 [ 8666.886887] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8666.896587] LustreError: 247825:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 8667.203311] Lustre: server umount tfs-MDT0001 complete [ 8680.786991] Lustre: server umount tfs-OST0000 complete [ 8694.859906] Lustre: server umount tfs-OST0001 complete [ 8721.922151] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8733.198604] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8733.800862] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8733.813565] Lustre: Skipped 3 previous similar messages [ 8739.745188] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8750.655206] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8756.271649] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8760.140548] Lustre: 253087:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 8768.030781] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8775.275587] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8778.611209] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49731 to 0x280000401:49824) [ 8786.671484] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8792.593141] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:35 to 0x2c0000401:97) [ 8794.216638] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8802.737502] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 8807.345783] Lustre: 254955:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 8815.287689] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 04:38:28 (1786696708) [ 8832.993365] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8833.003970] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8833.023608] Lustre: Skipped 3 previous similar messages [ 8833.032062] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8837.719931] Lustre: server umount lustre-MDT0000 complete [ 8838.630487] LustreError: 251942:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8838.652207] LustreError: 251942:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 12 previous similar messages [ 8841.740777] LustreError: 251921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696737 with bad export cookie 4186175391495353381 [ 8841.743420] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8841.754940] LustreError: 251921:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 8848.360424] Lustre: server umount lustre-MDT0001 complete [ 8858.952462] Lustre: server umount lustre-OST0000 complete [ 8869.627789] Lustre: server umount lustre-OST0001 complete [ 8877.444015] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 8890.119860] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 8936.897354] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 8936.897354] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8936.897354] [ 8936.943591] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8937.296758] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8937.326508] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8937.433753] Lustre: lustre-MDT0000: new disk, initializing [ 8937.533620] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8937.544534] Lustre: Skipped 3 previous similar messages [ 8937.560504] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8942.924254] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8955.970597] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 8955.970597] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 8955.970597] [ 8956.010178] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 8956.245684] Lustre: 258145:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8956.370098] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8956.374279] Lustre: Skipped 1 previous similar message [ 8956.467927] Lustre: lustre-MDT0001: new disk, initializing [ 8956.718661] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8956.743333] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8961.930582] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8973.504963] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8973.827943] Lustre: lustre-OST0000: new disk, initializing [ 8973.834166] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8973.843444] Lustre: 259728:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8975.287318] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8975.317067] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8975.429177] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8981.801782] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 8995.089408] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8995.394460] Lustre: lustre-OST0001: new disk, initializing [ 8995.402215] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 8995.411288] Lustre: 260600:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8997.440151] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 8997.475386] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 8997.621657] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 9002.562273] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9015.299033] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 04:41:48 (1786696908) [ 9034.439282] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 04:42:08 (1786696928) [ 9052.137459] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9052.143467] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9052.151712] LustreError: Skipped 1 previous similar message [ 9052.153675] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9052.179354] Lustre: Skipped 6 previous similar messages [ 9052.197232] Lustre: Skipped 9 previous similar messages [ 9055.832194] Lustre: server umount lustre-OST0000 complete [ 9064.420759] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9066.664279] Lustre: server umount lustre-MDT0000 complete [ 9071.161827] LustreError: 258136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786696966 with bad export cookie 4186175391495354928 [ 9071.170337] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9071.182840] LustreError: 258136:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9077.676543] Lustre: server umount lustre-MDT0001 complete [ 9091.108885] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 9163.898899] Lustre: server umount lustre-OST0001 complete [ 9171.734162] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 9187.887676] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 9229.393212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9229.690243] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9229.745109] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9229.906275] Lustre: lustre-MDT0000: new disk, initializing [ 9230.041500] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9230.051569] Lustre: Skipped 3 previous similar messages [ 9230.127781] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9236.681351] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9250.176378] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9250.347616] Lustre: 265407:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9250.475899] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9250.491579] Lustre: Skipped 1 previous similar message [ 9250.642865] Lustre: lustre-MDT0001: new disk, initializing [ 9250.945684] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9250.964185] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9255.268646] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9265.622899] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9272.120759] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9279.586176] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9279.920403] Lustre: lustre-OST0000: new disk, initializing [ 9279.933580] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9279.953763] Lustre: 267374:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9281.359431] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9281.374754] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9281.478501] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9287.203326] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9296.822751] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9307.113561] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9307.117077] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9307.120142] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9307.120150] Lustre: Skipped 7 previous similar messages [ 9307.143333] Lustre: Skipped 5 previous similar messages [ 9307.153795] LustreError: Skipped 1 previous similar message [ 9310.104674] Lustre: server umount lustre-OST0000 complete [ 9320.638676] Lustre: server umount lustre-MDT0000 complete [ 9322.464953] LustreError: 267458:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9322.486415] LustreError: 267458:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 14 previous similar messages [ 9325.293134] LustreError: 265400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786697220 with bad export cookie 4186175391495356405 [ 9325.302088] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9325.307948] LustreError: 265400:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9325.747872] Lustre: server umount lustre-MDT0001 complete [ 9337.991603] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 9340.995859] Key type lgssc unregistered [ 9341.270818] LNet: 269140:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9341.289527] LNetError: 269140:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9342.313803] LNet: Removed LNI 192.168.204.109@tcp [ 9343.342218] Key type .llcrypt unregistered [ 9343.360827] Key type ._llcrypt unregistered [ 9375.912771] Key type ._llcrypt registered [ 9375.914389] Key type .llcrypt registered [ 9376.019968] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 9377.406563] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9377.454153] alg: No test for adler32 (adler32-zlib) [ 9378.472683] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 9378.670441] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 9380.383233] Key type lgssc registered [ 9381.846123] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9390.166330] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9390.205355] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9391.680390] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9396.653815] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9401.837634] LustreError: 270608:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9401.862206] LustreError: 270608:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 9406.114279] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9406.638721] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9411.926327] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9420.060404] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9426.564225] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9435.642979] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9436.224804] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9437.260906] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 9443.414589] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9451.634173] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9452.973666] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 9456.618290] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9456.634595] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 9456.636279] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 9456.667366] Lustre: Skipped 1 previous similar message [ 9457.499628] Lustre: server umount lustre-OST0000 complete [ 9462.762797] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 9462.783984] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9462.806198] Lustre: Skipped 1 previous similar message [ 9462.812162] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9466.851619] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9466.852723] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9468.374348] Lustre: server umount lustre-MDT0000 complete [ 9471.970730] LustreError: 270604:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9471.993287] LustreError: 270604:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 9472.301228] LustreError: 270589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786697367 with bad export cookie 72051066724111971 [ 9472.309671] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9472.313021] LustreError: 270589:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 9472.730135] Lustre: server umount lustre-MDT0001 complete [ 9485.237389] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 9489.143559] Key type lgssc unregistered [ 9489.594621] LNet: 274193:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9489.613555] LNetError: 274193:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9489.653947] LNet: Removed LNI 192.168.204.109@tcp [ 9491.164631] Key type .llcrypt unregistered [ 9491.166888] Key type ._llcrypt unregistered [ 9511.428897] Key type ._llcrypt registered [ 9511.431836] Key type .llcrypt registered [ 9511.628349] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 9512.829956] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9512.842770] alg: No test for adler32 (adler32-zlib) [ 9513.866163] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 9514.104771] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 9515.807711] Key type lgssc registered [ 9517.202612] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9526.764331] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9526.778727] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9542.303716] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 9547.423555] LustreError: 275390:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.204.109@tcp: failed processing log, type 4: rc = -110 [ 9574.113606] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9582.697337] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9592.433167] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9597.264664] Lustre: server umount lustre-OST0000 complete [ 9617.724777] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [ 9620.366590] Key type lgssc unregistered [ 9620.639571] LNet: 277026:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 9620.649821] LNetError: 277026:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 9621.680862] LNet: Removed LNI 192.168.204.109@tcp [ 9622.921346] Key type .llcrypt unregistered [ 9622.924161] Key type ._llcrypt unregistered [ 9677.884872] Key type ._llcrypt registered [ 9677.889482] Key type .llcrypt registered [ 9678.033567] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 9678.867218] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 9678.889812] alg: No test for adler32 (adler32-zlib) [ 9679.934840] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [ 9680.167971] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [ 9681.913050] Key type lgssc registered [ 9683.113375] Lustre: Echo OBD driver; http://www.lustre.org/ [ 9697.148433] loop0: detected capacity change from 0 to 204800000 [ 9697.353463] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9697.377919] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9697.734247] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9698.872163] loop0: detected capacity change from 0 to 204800000 [ 9699.078127] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9699.096107] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9699.560833] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9700.783474] loop0: detected capacity change from 0 to 204800000 [ 9701.008856] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9701.015942] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9701.080475] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9702.099436] loop0: detected capacity change from 0 to 204800000 [ 9702.273643] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9702.392486] print_req_error: 1 callbacks suppressed [ 9702.397770] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 9703.468340] loop0: detected capacity change from 0 to 204800000 [ 9703.511140] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9707.335922] loop0: detected capacity change from 0 to 204800000 [ 9707.365690] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9711.716674] loop0: detected capacity change from 0 to 204800000 [ 9711.771643] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9716.850125] loop0: detected capacity change from 0 to 204800000 [ 9716.896041] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 9721.517254] loop0: detected capacity change from 0 to 204800000 [ 9721.678302] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 9721.764960] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9721.833513] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9721.840598] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 9721.853601] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9721.868324] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 9721.879365] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9721.896496] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 9721.927145] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 9721.947679] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9721.968805] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 9721.993039] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 9722.012848] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 9722.026609] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 9722.036903] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 9722.049186] Lustre: 280453:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 9729.005577] Lustre: server umount MGS complete [ 9730.160394] loop0: detected capacity change from 0 to 204800000 [ 9730.216524] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9730.801615] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9730.914804] Lustre: lustre-MDT0000: Aborting client recovery [ 9732.093887] loop1: detected capacity change from 0 to 204800000 [ 9732.202049] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9732.297172] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9732.311748] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 9732.322772] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9732.348335] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 9732.364762] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9732.386564] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9732.415976] Lustre: 280887:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 9732.464916] Lustre: Found index 1 for lustre-MDT0001, updating log [ 9732.741392] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9732.810789] Lustre: lustre-MDT0001: Aborting client recovery [ 9734.276295] loop2: detected capacity change from 0 to 204800000 [ 9734.410823] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9734.538047] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9734.559951] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 9734.584340] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 9734.624084] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9734.632679] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9734.645851] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9734.664887] Lustre: 280967:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 9734.753896] Lustre: Found index 0 for lustre-OST0000, updating log [ 9734.790790] Lustre: 280986:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9734.895114] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9734.919416] Lustre: lustre-OST0000: Aborting recovery [ 9735.864206] loop3: detected capacity change from 0 to 204800000 [ 9735.943302] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9736.088093] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 9736.103251] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 9736.120874] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 9736.129884] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 9736.152982] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 9736.173749] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 9736.181921] Lustre: 281051:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 9736.198544] Lustre: Found index 1 for lustre-OST0001, updating log [ 9736.230060] Lustre: 281056:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9736.320709] Lustre: lustre-OST0001: Aborting recovery [ 9740.392698] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9740.423410] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 9740.439294] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 9740.440840] LustreError: 280831:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 10, retries 0, failed: rc = -108 [ 9740.456832] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 9742.437547] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 9742.438172] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 9744.535243] LustreError: 280798:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 1138us: rc = -2 [ 9745.058924] LustreError: 280797:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 4793us: rc = -2 [ 9745.092328] LustreError: 280797:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 18 previous similar messages [ 9763.298783] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9763.305251] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 9763.323885] Lustre: Skipped 3 previous similar messages [ 9763.336478] Lustre: Skipped 2 previous similar messages [ 9764.724128] Lustre: server umount lustre-MDT0000 complete [ 9765.640301] LustreError: 280782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786697661 with bad export cookie 4008847094878285604 [ 9765.659552] LustreError: 280782:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 9765.663731] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 9768.419389] LustreError: 280796:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9768.420215] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 9768.422236] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9768.422243] Lustre: Skipped 1 previous similar message [ 9768.437649] LustreError: 280796:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 9768.498772] Lustre: Skipped 1 previous similar message [ 9770.468439] LustreError: 280797:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 9770.468971] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 9770.499209] Lustre: Skipped 2 previous similar messages [ 9772.037514] Lustre: server umount lustre-MDT0001 complete [ 9787.359273] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9787.483987] Lustre: server umount lustre-OST0000 complete [ 9803.231236] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 9803.499501] Lustre: server umount lustre-OST0001 complete [ 9814.098288] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 04:55:07 (1786697707) [ 9848.558147] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [ 9861.721425] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [ 9899.756037] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9900.051120] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 9900.103489] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 9900.199933] Lustre: lustre-MDT0000: new disk, initializing [ 9900.315932] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 9900.329389] Lustre: Skipped 1 previous similar message [ 9900.356893] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 9905.409817] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9917.443336] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 9917.560926] Lustre: 285124:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 9917.612993] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 9917.618232] Lustre: Skipped 1 previous similar message [ 9917.690228] Lustre: lustre-MDT0001: new disk, initializing [ 9917.786468] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 9917.834482] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 9917.851248] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 9923.295968] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9933.508860] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 9939.959044] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 9948.676541] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 9949.186970] Lustre: lustre-OST0000: new disk, initializing [ 9949.197878] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 9949.207222] Lustre: 287096:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 9949.375393] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 9955.422933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 9955.455225] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 9955.720697] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 9958.828235] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [ 9972.513547] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 9978.742792] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 9990.535796] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 9998.193621] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10008.615880] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [10047.972173] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10047.972848] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10047.974669] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10047.988961] Lustre: Skipped 1 previous similar message [10053.093066] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10053.110037] Lustre: Skipped 2 previous similar messages [10054.302888] Lustre: server umount lustre-OST0000 complete [10061.281852] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10061.306450] LustreError: Skipped 1 previous similar message [10061.316078] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10061.336084] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10066.537384] Lustre: server umount lustre-MDT0000 complete [10068.459358] LustreError: 285138:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10068.479081] LustreError: 285138:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10071.337121] LustreError: 285920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786697966 with bad export cookie 4008847094878288334 [10071.345782] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10071.347727] LustreError: 285920:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10071.837665] Lustre: server umount lustre-MDT0001 complete [10082.827488] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10089.354413] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10095.655928] Lustre: server umount MGS complete [10109.333077] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10109.741745] LustreError: 291019:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10109.770976] LustreError: 291032:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10109.836208] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10115.043688] LustreError: 291020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10115.110227] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10124.664852] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10130.464804] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10139.385359] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10147.809595] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10157.503800] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10158.033761] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10158.042016] Lustre: Skipped 1 previous similar message [10167.199240] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10175.238981] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10210.272678] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [10210.285325] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10210.294922] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10210.329560] Lustre: Skipped 2 previous similar messages [10210.335916] Lustre: Skipped 2 previous similar messages [10213.398519] Lustre: server umount lustre-OST0000 complete [10220.513599] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10223.938974] Lustre: server umount lustre-MDT0000 complete [10225.634496] LustreError: 291020:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10225.662833] LustreError: 291020:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [10228.746787] LustreError: 293761:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786698124 with bad export cookie 4008847094878290287 [10228.764898] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10229.675941] Lustre: server umount lustre-MDT0001 complete [10244.146343] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [10247.866099] Key type lgssc unregistered [10248.188930] LNet: 294981:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10248.205649] LNetError: 294981:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10248.230915] LNet: Removed LNI 192.168.204.109@tcp [10249.326650] Key type .llcrypt unregistered [10249.332423] Key type ._llcrypt unregistered [10299.520639] Key type ._llcrypt registered [10299.525597] Key type .llcrypt registered [10299.647733] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [10319.063109] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [10320.593896] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10320.828750] alg: No test for adler32 (adler32-zlib) [10322.133708] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [10322.550784] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [10324.455976] Key type lgssc registered [10326.706610] Lustre: Echo OBD driver; http://www.lustre.org/ [10375.470294] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10375.511492] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10376.820338] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10376.867255] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10377.001397] Lustre: lustre-MDT0000: new disk, initializing [10377.167292] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10377.202622] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10382.650219] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10396.477083] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10396.690577] Lustre: 298945:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [10396.800612] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [10396.804273] Lustre: Skipped 1 previous similar message [10396.942431] Lustre: lustre-MDT0001: new disk, initializing [10397.107898] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10397.148559] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [10397.163512] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [10403.149321] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10414.493197] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10422.758788] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10431.609510] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10432.009416] Lustre: lustre-OST0000: new disk, initializing [10432.013979] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10432.020151] Lustre: 300918:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10432.163951] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10440.358647] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10441.348362] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [10441.364793] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [10441.615103] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [10451.409419] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10458.377710] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [10471.350376] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [10476.261892] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [10528.742175] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10528.754512] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10528.765282] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10528.766796] Lustre: Skipped 1 previous similar message [10530.646831] Lustre: server umount lustre-OST0000 complete [10535.399328] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [10535.413839] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10535.438534] Lustre: Skipped 1 previous similar message [10535.448709] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10538.987566] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10539.012365] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10540.922032] Lustre: server umount lustre-MDT0000 complete [10545.046387] LustreError: 298938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786698440 with bad export cookie 7169369523527187456 [10545.049663] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10545.065111] LustreError: 298938:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [10545.553399] Lustre: server umount lustre-MDT0001 complete [10555.286556] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10560.539553] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10567.181533] Lustre: server umount MGS complete [10580.744372] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10581.213976] LustreError: 304737:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10581.242405] LustreError: 304737:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [10581.259504] LustreError: 304750:0:(qmt_pool.c:1639:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [10581.327410] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10586.180814] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10586.594292] LustreError: 304738:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10590.699985] LustreError: 304737:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10595.676927] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10595.809401] LustreError: 304738:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [10596.149637] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [10600.828405] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10610.263246] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [10617.847956] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [10627.186994] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10627.499216] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10635.359788] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10645.479090] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [10681.824472] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [10681.827977] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10681.845470] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [10681.861826] Lustre: Skipped 1 previous similar message [10681.877529] Lustre: Skipped 2 previous similar messages [10686.178364] Lustre: server umount lustre-OST0000 complete [10692.068775] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [10692.072226] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [10692.086332] Lustre: Skipped 1 previous similar message [10692.105197] Lustre: Skipped 1 previous similar message [10697.480749] Lustre: server umount lustre-MDT0000 complete [10701.177263] LustreError: 304717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786698596 with bad export cookie 7169369523527189297 [10701.179062] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [10701.183816] LustreError: 304717:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [10701.515911] Lustre: server umount lustre-MDT0001 complete [10716.302937] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [10720.370766] Key type lgssc unregistered [10720.733245] LNet: 308691:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [10720.761221] LNetError: 308691:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [10720.786058] LNet: Removed LNI 192.168.204.109@tcp [10721.818797] Key type .llcrypt unregistered [10721.821869] Key type ._llcrypt unregistered [10745.782384] Key type ._llcrypt registered [10745.788661] Key type .llcrypt registered [10745.889232] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [10761.725191] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [10763.240641] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [10763.304555] alg: No test for adler32 (adler32-zlib) [10764.382199] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [10764.627088] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [10766.305928] Key type lgssc registered [10767.624395] Lustre: Echo OBD driver; http://www.lustre.org/ [10795.309863] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [10812.629573] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [10812.655036] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [10813.879326] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [10813.911077] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [10813.997326] Lustre: lustre-MDT0000: new disk, initializing [10814.091839] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [10814.117365] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [10818.991536] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [10832.433146] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [10832.689972] Lustre: lustre-OST0000: new disk, initializing [10832.693417] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [10832.698330] Lustre: Skipped 1 previous similar message [10832.703807] Lustre: 312742:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [10832.775571] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [10838.730763] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [10838.753873] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [10838.792869] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [10840.780434] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [11883.492322] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [11883.509394] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [11885.253359] Lustre: server umount lustre-MDT0000 complete [11890.434279] LustreError: 311930:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786699786 with bad export cookie 7038157607385296530 [11890.460417] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [11890.809968] Lustre: server umount lustre-OST0000 complete [11923.837936] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [11935.065835] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [11977.654245] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11977.918156] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [11977.985788] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [11978.092888] Lustre: lustre-MDT0000: new disk, initializing [11978.214946] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [11978.250491] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [11984.572086] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [11997.787665] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [11997.963678] Lustre: 317112:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [11998.044672] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [11998.050612] Lustre: Skipped 1 previous similar message [11998.148659] Lustre: lustre-MDT0001: new disk, initializing [11998.282099] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [11998.327815] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [11998.344025] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [12003.963421] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12014.873037] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12020.147890] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12027.763924] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12028.038849] Lustre: lustre-OST0000: new disk, initializing [12028.043802] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [12028.053102] Lustre: 319081:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12028.143952] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12035.140836] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12037.746668] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [12037.762187] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [12037.860627] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [12044.758094] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12049.302422] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12049.601991] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12053.986776] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [12054.159713] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12058.593586] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12058.602722] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12058.616993] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12058.620452] Lustre: Skipped 1 previous similar message [12063.728468] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12063.735379] Lustre: Skipped 1 previous similar message [12068.843779] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12068.856984] Lustre: Skipped 1 previous similar message [12070.367141] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [12070.604842] Lustre: server umount lustre-OST0000 complete [12074.976522] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12074.994649] LustreError: Skipped 1 previous similar message [12074.996491] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12075.022625] Lustre: Skipped 1 previous similar message [12075.031116] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12079.077082] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12080.667433] Lustre: server umount lustre-MDT0000 complete [12084.505263] LustreError: 317103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786699980 with bad export cookie 7038157607388237349 [12084.519293] LustreError: 317103:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [12084.521557] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12084.787890] Lustre: server umount lustre-MDT0001 complete [12087.572600] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [12088.909576] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 05:33:03 (1786699983) [12094.119737] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12094.709034] LustreError: 321395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12094.721498] LustreError: 321395:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12094.808695] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12098.381402] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12100.065280] LustreError: 321395:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12105.188152] LustreError: 321396:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12105.489129] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12105.830130] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12109.926734] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12116.360470] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12121.995590] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [12129.163116] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12129.485908] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12135.147624] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12141.814566] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12148.055078] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12148.212953] Lustre: lustre-OST0001: new disk, initializing [12148.216121] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [12148.220934] Lustre: 324230:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [12148.295706] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12154.212430] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12158.010254] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [12158.019842] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [12158.080907] LustreError: 324750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12159.141446] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12159.153153] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 2 previous similar messages [12160.162995] LustreError: 324750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12162.217145] LustreError: 324750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [12162.226180] LustreError: 324750:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [12163.417440] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [12163.629216] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [12166.371286] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12166.379393] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 8 previous similar messages [12169.313777] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffffa0a890cb3800.ost_server_uuid 50 [12170.809484] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffffa0a890cb3800.ost_server_uuid in FULL state after 0 sec [12174.570817] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [12174.582734] LustreError: 324759:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 15 previous similar messages [12176.072968] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffffa0a890cb3800.ost_server_uuid 50 [12177.463684] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffffa0a890cb3800.ost_server_uuid in FULL state after 0 sec [12179.772347] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [12194.289256] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12194.293326] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12194.307949] Lustre: Skipped 1 previous similar message [12194.311607] Lustre: Skipped 2 previous similar messages [12198.524562] Lustre: server umount lustre-OST0001 complete [12199.399585] LustreError: 323215:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12199.411643] LustreError: 323215:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12204.516292] LustreError: 323217:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12204.516481] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12204.533495] LustreError: 323217:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12204.548482] Lustre: Skipped 1 previous similar message [12210.022766] Lustre: server umount lustre-OST0000 complete [12214.758451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12214.765953] Lustre: Skipped 7 previous similar messages [12219.718649] Lustre: server umount lustre-MDT0000 complete [12219.874705] LustreError: 321396:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12219.887513] LustreError: 321396:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12222.730516] LustreError: 322128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786700118 with bad export cookie 7038157607388238231 [12222.736596] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12222.739829] LustreError: 322128:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12223.000665] Lustre: server umount lustre-MDT0001 complete [12233.601631] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [12236.585475] Key type lgssc unregistered [12236.926095] LNet: 326923:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12236.935577] LNetError: 326923:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12236.948820] LNet: Removed LNI 192.168.204.109@tcp [12237.882767] Key type .llcrypt unregistered [12237.884919] Key type ._llcrypt unregistered [12262.356221] Key type ._llcrypt registered [12262.360840] Key type .llcrypt registered [12262.517875] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [12263.615229] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12263.684932] alg: No test for adler32 (adler32-zlib) [12264.835956] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [12264.994643] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [12266.639428] Key type lgssc registered [12267.833542] Lustre: Echo OBD driver; http://www.lustre.org/ [12276.738364] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12276.776799] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12278.264236] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12282.150487] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12287.460351] LustreError: 328340:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12287.485991] LustreError: 328340:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12289.464898] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [12292.577628] LustreError: 328341:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12295.771414] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12296.142505] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12299.982564] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12307.771961] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12308.109324] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12313.812464] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12320.166651] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12320.362761] LustreError: 330022:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12322.401318] LustreError: 330023:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12322.421993] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [12322.427688] LustreError: 330023:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12327.005384] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12327.233833] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12332.532056] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [12333.270385] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12339.721437] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [12347.119610] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [12347.310559] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [12455.395281] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [12455.399251] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12455.406427] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [12455.417096] Lustre: Skipped 1 previous similar message [12457.319127] Lustre: server umount lustre-OST0001 complete [12460.517670] LustreError: 330781:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12460.544729] LustreError: 330781:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12464.100514] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [12464.108697] LustreError: Skipped 1 previous similar message [12464.115640] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12464.132571] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12464.136844] Lustre: Skipped 1 previous similar message [12465.633067] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12465.649528] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [12468.752492] Lustre: server umount lustre-OST0000 complete [12475.362393] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [12475.372463] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12475.393647] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12475.395692] Lustre: Skipped 1 previous similar message [12478.324968] Lustre: server umount lustre-MDT0000 complete [12480.994062] LustreError: 328340:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12481.010864] LustreError: 328340:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12481.459633] LustreError: 328321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786700377 with bad export cookie 9061472989362015980 [12481.463503] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12481.472972] LustreError: 328321:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [12481.791723] Lustre: server umount lustre-MDT0001 complete [12491.815360] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [12494.402137] Key type lgssc unregistered [12494.697219] LNet: 333291:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [12494.708513] LNetError: 333291:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [12494.728920] LNet: Removed LNI 192.168.204.109@tcp [12495.512198] Key type .llcrypt unregistered [12495.515799] Key type ._llcrypt unregistered [12537.813674] Key type ._llcrypt registered [12537.818274] Key type .llcrypt registered [12537.932537] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [12538.974237] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [12539.179456] alg: No test for adler32 (adler32-zlib) [12540.247340] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [12540.524896] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [12542.183207] Key type lgssc registered [12543.383724] Lustre: Echo OBD driver; http://www.lustre.org/ [12551.953322] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [12551.967906] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12553.519254] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12557.920425] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12563.941899] LustreError: 335580:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12563.958095] LustreError: 335580:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [12566.402367] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12566.801973] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12571.480991] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12575.052037] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [12582.494715] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12582.795344] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [12590.286288] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12592.057879] LustreError: 337068:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12597.239690] LustreError: 337498:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12599.692991] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12599.932618] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12600.975577] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [12601.008109] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [12607.105931] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12616.247792] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12626.109070] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [12637.168345] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12637.178779] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12637.179346] Lustre: Skipped 1 previous similar message [12637.190922] Lustre: Skipped 3 previous similar messages [12642.223830] Lustre: server umount lustre-MDT0000 complete [12642.278813] LustreError: 335575:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12642.300179] LustreError: 335575:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [12645.413649] LustreError: 335560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786700541 with bad export cookie 7252928376207288752 [12645.427114] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12645.429267] LustreError: 335560:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [12645.781539] Lustre: server umount lustre-MDT0001 complete [12660.075952] Lustre: server umount lustre-OST0000 complete [12674.285627] Lustre: server umount lustre-OST0001 complete [12688.140543] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [12696.354119] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12696.697138] LustreError: 340846:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12696.754671] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12701.388352] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12707.306106] LustreError: 340847:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12707.321636] LustreError: 340847:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [12709.136676] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12709.506329] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [12713.591666] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12716.566371] Lustre: 341986:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [12722.921847] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12729.256161] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12734.503449] LustreError: 342342:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [12736.778068] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [12736.962430] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [12736.967793] Lustre: Skipped 1 previous similar message [12742.079377] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [12742.084342] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [12742.617503] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12749.993078] Lustre: DEBUG MARKER: Using TIMEOUT=20 [12753.446836] Lustre: 343862:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [12757.476080] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12757.484178] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12757.497386] Lustre: Skipped 5 previous similar messages [12757.500611] Lustre: Skipped 2 previous similar messages [12762.600065] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [12762.605841] Lustre: Skipped 4 previous similar messages [12763.440771] Lustre: server umount lustre-MDT0000 complete [12767.117921] LustreError: 340827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786700662 with bad export cookie 7252928376207290446 [12767.125533] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [12767.130501] LustreError: 340827:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [12767.503940] Lustre: server umount lustre-MDT0001 complete [12782.062502] Lustre: server umount lustre-OST0000 complete [12784.095172] Lustre: 334843:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1786700663/real 1786700663] req@ffff88d903b15180 x1873491196489856/t0(0) o400->lustre-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1786700679 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [12784.138988] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [12785.546204] Lustre: server umount lustre-OST0001 complete [12789.076377] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [12790.636486] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [12792.199512] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 05:44:46 (1786700686) [12816.592655] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [12826.359598] loop0: detected capacity change from 0 to 1099511627776 [12826.943721] SGI XFS with ACLs, security attributes, quota, no debug enabled [12826.973294] *** VALIDATE xfs *** [12826.983248] XFS (loop0): Mounting V5 Filesystem [12827.000372] XFS (loop0): Ending clean mount [12828.378366] loop1: detected capacity change from 0 to 18691697672192 [12933.168597] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [12942.021665] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [12983.458120] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12983.741748] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [12983.777292] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [12983.872375] Lustre: lustre-MDT0000: new disk, initializing [12983.919869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [12983.949330] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [12988.591340] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [12998.985648] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [12999.110924] Lustre: 348996:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [12999.157660] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [12999.160792] Lustre: Skipped 1 previous similar message [12999.216668] Lustre: lustre-MDT0001: new disk, initializing [12999.311222] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [12999.317170] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [13003.653814] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13013.933115] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13019.920812] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13027.352563] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13027.785521] Lustre: lustre-OST0000: new disk, initializing [13027.791547] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [13027.799631] Lustre: 350964:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [13029.131892] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [13029.152469] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [13029.272105] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [13035.303830] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13047.208342] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13053.842631] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [13054.173085] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [13059.902827] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [13060.167481] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [13065.184175] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13065.186928] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13065.190241] LustreError: Skipped 1 previous similar message [13065.191347] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13065.227246] Lustre: Skipped 1 previous similar message [13068.706927] Lustre: server umount lustre-OST0000 complete [13075.425124] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13075.457790] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13075.477590] Lustre: Skipped 2 previous similar messages [13084.651658] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [13084.671399] Lustre: Skipped 3 previous similar messages [13087.200098] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [13087.530308] Lustre: server umount lustre-MDT0000 complete [13089.769159] LustreError: 349003:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13089.808989] LustreError: 349003:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13092.063524] LustreError: 348989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786700987 with bad export cookie 7252928376207292000 [13092.070611] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13092.079299] LustreError: 348989:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [13092.603502] Lustre: server umount lustre-MDT0001 complete [13104.016532] XFS (loop0): Unmounting Filesystem [13108.841141] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 05:50:02 (1786701002) [13116.033331] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13116.526366] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13116.533413] Lustre: Skipped 2 previous similar messages [13121.698596] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13130.161372] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13135.272978] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13144.201959] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13150.947963] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13158.473403] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13165.916785] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13173.907636] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13187.557490] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [13187.566860] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13187.578181] Lustre: Skipped 1 previous similar message [13187.591515] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [13191.651187] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [13193.259854] Lustre: server umount lustre-OST0000 complete [13205.822697] Lustre: server umount lustre-MDT0000 complete [13207.016035] LustreError: 353521:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13207.039865] LustreError: 353521:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [13210.437237] LustreError: 353507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786701106 with bad export cookie 7252928376207292882 [13210.439749] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [13210.450622] LustreError: 353507:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 5 previous similar messages [13210.866989] Lustre: server umount lustre-MDT0001 complete [13223.555789] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [13226.607725] Key type lgssc unregistered [13226.951686] LNet: 357163:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [13226.962964] LNetError: 357163:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [13228.007084] LNet: Removed LNI 192.168.204.109@tcp [13229.150200] Key type .llcrypt unregistered [13229.157394] Key type ._llcrypt unregistered [13258.146663] Key type ._llcrypt registered [13258.149812] Key type .llcrypt registered [13258.310747] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [13259.658208] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [13259.732198] alg: No test for adler32 (adler32-zlib) [13260.798245] Lustre: Lustre: Build Version: 2.17.57_1_g9a8e296 [13261.151083] LNet: Added LNI 192.168.204.109@tcp [8/256/0/180] [13262.871244] Key type lgssc registered [13264.198421] Lustre: Echo OBD driver; http://www.lustre.org/ [13272.559383] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [13272.576458] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13274.262051] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [13279.609739] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13283.818338] LustreError: 358732:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13283.834269] LustreError: 358732:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13288.932841] LustreError: 358733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13289.070072] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13289.533874] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13293.984170] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13302.718577] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [13310.590659] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [13318.969349] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [13319.171193] LustreError: 360557:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13319.521791] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [13326.624066] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13335.121949] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [13342.022519] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.9@tcp (stopping) [13345.248969] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13345.279584] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13345.297547] Lustre: Skipped 1 previous similar message [13347.116189] Lustre: lustre-MDT0001: Not available for connect from 192.168.204.9@tcp (stopping) [13348.010622] Lustre: server umount lustre-MDT0001 complete [13350.379775] LustreError: 358729:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13350.403666] LustreError: 358729:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [13355.501940] LustreError: 358733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13355.532941] LustreError: 358733:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13359.413812] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13359.584511] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13359.608762] Lustre: Found index 1 for lustre-MDT0001, updating log [13359.693523] Lustre: 358722:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13360.118784] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [13365.240303] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13365.257961] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13368.305890] LustreError: 362333:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13368.316601] LustreError: 362333:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13368.341238] LustreError: 358725:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13368.438688] LustreError: 362336:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.109@tcp: cfg command failed: rc = -17 [13368.447427] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [13368.447427] [13672.418606] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [13672.420740] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [13672.433869] Lustre: Skipped 1 previous similar message [13672.436461] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [13672.456504] Lustre: Skipped 1 previous similar message [13677.536900] LustreError: 358733:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [13677.553158] Lustre: server umount lustre-MDT0001 complete [13677.557883] LustreError: 358733:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [13688.324296] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [13688.426100] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [13688.430746] Lustre: Found index 1 for lustre-MDT0001, updating log [13688.486613] Lustre: 358722:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [13688.927675] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [13693.733595] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [13693.938257] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [13697.532842] LustreError: 370506:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [13697.542488] LustreError: 370506:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [13697.550070] LustreError: 358725:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [13697.557726] LustreError: 358725:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [13697.617690] LustreError: 370509:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [13697.623973] LustreError: 370509:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [13697.630395] LustreError: 370509:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.109@tcp: cfg command failed: rc = -17 [13697.641639] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.204.109@tcp [13697.641639] [14001.136384] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14001.144920] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [14001.162411] Lustre: Skipped 1 previous similar message [14001.164934] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [14001.164941] Lustre: Skipped 1 previous similar message [14005.572063] Lustre: server umount lustre-MDT0001 complete [14006.264732] LustreError: 358729:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14006.298550] LustreError: 358729:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [14017.265139] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14017.482510] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [14017.489360] Lustre: Found index 1 for lustre-MDT0001, updating log [14017.556383] Lustre: 358723:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14017.970777] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [14023.115502] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [14023.157711] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [14023.164461] LustreError: 378641:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [14023.180570] LustreError: 378641:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [14023.196109] LustreError: 358725:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [14023.209741] LustreError: 358725:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [14023.274113] LustreError: 378649:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [14023.298312] LustreError: 378649:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.204.109@tcp: cfg command failed: rc = -114 [14023.313993] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [14023.313993] [14350.819360] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14350.831838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14350.834993] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14350.835023] Lustre: Skipped 2 previous similar messages [14350.839510] Lustre: Skipped 2 previous similar messages [14353.102218] Lustre: server umount lustre-MDT0000 complete [14355.938734] LustreError: 362179:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14355.967518] LustreError: 362179:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 8 previous similar messages [14357.137033] LustreError: 358713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786702252 with bad export cookie 9378804543683966873 [14357.151643] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14357.161344] LustreError: 358713:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [14361.062163] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14361.077913] Lustre: Skipped 1 previous similar message [14363.571581] Lustre: server umount lustre-MDT0001 complete [14368.697688] Lustre: server umount lustre-OST0000 complete [14379.458600] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [14391.702327] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [14430.277178] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 06:12:04 (1786702324) [14438.164797] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14438.413984] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [14438.439984] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [14438.516915] Lustre: lustre-MDT0000: new disk, initializing [14438.615659] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [14438.648468] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [14443.454690] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [14455.687540] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [14455.908865] Lustre: 389181:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [14456.023828] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [14456.032798] Lustre: Skipped 1 previous similar message [14456.115989] Lustre: lustre-MDT0001: new disk, initializing [14456.245311] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [14456.277380] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [14456.285976] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [14460.362980] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [14469.638575] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [14475.851598] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [14482.864165] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [14483.100053] Lustre: lustre-OST0000: new disk, initializing [14483.102692] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [14483.107653] Lustre: 391148:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [14483.205345] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [14488.633041] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [14488.643194] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [14488.743741] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [14490.026327] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_default_debug -1 all [14499.025987] Lustre: DEBUG MARKER: oleg409-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [14507.502896] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [14507.518102] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [14507.537882] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [14507.546425] Lustre: Skipped 4 previous similar messages [14508.971501] Lustre: server umount lustre-MDT0000 complete [14509.537054] LustreError: 389192:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [14509.554750] LustreError: 389192:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [14512.888415] LustreError: 389172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1786702408 with bad export cookie 9378804543683968826 [14512.905314] LustreError: MGC192.168.204.109@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [14512.915204] LustreError: 389172:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [14512.928566] LustreError: 389986:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [14513.319957] Lustre: server umount lustre-MDT0001 complete [14528.808294] Lustre: server umount lustre-OST0000 complete [14563.798390] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing set_hostid [14576.038482] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing load_modules_local [14607.799898] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 14324 sec ========= 06:15:01 (1786702501) [14609.613114] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 06:15:03 (1786702503) === [14612.991288] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 06:15:06 (1786702506) === [14659.998550] Lustre: DEBUG MARKER: oleg409-server.virtnet: executing unload_modules_local [14663.284688] Key type lgssc unregistered [14663.624500] LNet: 397519:0:(lib-ptl.c:964:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [14663.631465] LNetError: 397519:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [14664.678593] LNet: Removed LNI 192.168.204.109@tcp [14665.749148] Key type .llcrypt unregistered [14665.752520] Key type ._llcrypt unregistered