[ 0.000000] Linux version 4.18.0rh8.10-debug (green@maintenance) (gcc version 8.5.0 20210514 (Red Hat 8.5.0-26) (GCC)) #2 SMP Mon Jul 14 01:24:22 EDT 2025 [ 0.000000] Command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers' [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] signal: max sigframe size: 1776 [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bffcdfff] usable [ 0.000000] BIOS-e820: [mem 0x00000000bffce000-0x00000000bfffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x0000000146dfffff] usable [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] SMBIOS 2.8 present. [ 0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-10.fc44 06/10/2025 [ 0.000000] Hypervisor detected: KVM [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00 [ 0.000000] kvm-clock: using sched offset of 523099146 cycles [ 0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns [ 0.000000] tsc: Detected 2400.000 MHz processor [ 0.000000] last_pfn = 0x146e00 max_arch_pfn = 0x400000000 [ 0.000000] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT [ 0.000000] last_pfn = 0xbffce max_arch_pfn = 0x400000000 [ 0.000000] found SMP MP-table at [mem 0x000f54b0-0x000f54bf] [ 0.000000] RAMDISK: [mem 0xbcc54000-0xbffbffff] [ 0.000000] ACPI: Early table checksum verification disabled [ 0.000000] ACPI: RSDP 0x00000000000F52D0 000014 (v00 BOCHS ) [ 0.000000] ACPI: RSDT 0x00000000BFFE247C 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACP 0x00000000BFFE2318 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: DSDT 0x00000000BFFE0040 0022D8 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: FACS 0x00000000BFFE0000 000040 [ 0.000000] ACPI: APIC 0x00000000BFFE238C 000090 (v03 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: HPET 0x00000000BFFE241C 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: WAET 0x00000000BFFE2454 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001) [ 0.000000] ACPI: Reserving FACP table memory at [mem 0xbffe2318-0xbffe238b] [ 0.000000] ACPI: Reserving DSDT table memory at [mem 0xbffe0040-0xbffe2317] [ 0.000000] ACPI: Reserving FACS table memory at [mem 0xbffe0000-0xbffe003f] [ 0.000000] ACPI: Reserving APIC table memory at [mem 0xbffe238c-0xbffe241b] [ 0.000000] ACPI: Reserving HPET table memory at [mem 0xbffe241c-0xbffe2453] [ 0.000000] ACPI: Reserving WAET table memory at [mem 0xbffe2454-0xbffe247b] [ 0.000000] No NUMA configuration found [ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000146dfffff] [ 0.000000] NODE_DATA(0) allocated [mem 0x1465a3000-0x1465cdfff] [ 0.000000] Reserving 256MB of memory at 2752MB for crashkernel (System RAM: 4205MB) [ 0.000000] Zone ranges: [ 0.000000] DMA [mem 0x0000000000001000-0x0000000000ffffff] [ 0.000000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff] [ 0.000000] Normal [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Device empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000000001000-0x000000000009efff] [ 0.000000] node 0: [mem 0x0000000000100000-0x00000000bffcdfff] [ 0.000000] node 0: [mem 0x0000000100000000-0x0000000146dfffff] [ 0.000000] Zeroed struct page in unavailable ranges: 4756 pages [ 0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x0000000146dfffff] [ 0.000000] ACPI: PM-Timer IO Port: 0x608 [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) [ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23 [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level) [ 0.000000] Using ACPI (MADT) for SMP configuration information [ 0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000 [ 0.000000] TSC deadline timer available [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] kvm-guest: KVM setup pv remote TLB flush [ 0.000000] kvm-guest: setup PV sched yield [ 0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff] [ 0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff] [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff] [ 0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff] [ 0.000000] PM: Registered nosave memory: [mem 0xbffce000-0xbfffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfeffbfff] [ 0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff] [ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff] [ 0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff] [ 0.000000] [mem 0xc0000000-0xfeffbfff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on KVM [ 0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1 [ 0.000000] percpu: Embedded 63 pages/cpu s221184 r8192 d28672 u524288 [ 0.000000] kvm-guest: PV spinlocks enabled [ 0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1059606 [ 0.000000] Policy zone: Normal [ 0.000000] Kernel command line: rd.shell root=nbd:192.168.200.253:rocky8.10:ext4:ro:-p,-b4096 ro crashkernel=256M panic=1 nomodeset ipmtu=9000 ip=dhcp rd.neednet=1 init_on_free=off mitigations=off console=ttyS1,115200 audit=0 [ 0.000000] Specific versions of hardware are certified with Red Hat Enterprise Linux 8. Please see the list of hardware certified with Red Hat Enterprise Linux 8 at https://catalog.redhat.com. [ 0.000000] audit: disabled (until reboot) [ 0.000000] software IO TLB: area num 4. [ 0.000000] Memory: 2829652K/4306352K available (18435K kernel code, 11221K rwdata, 7248K rodata, 2908K init, 18040K bss, 524580K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] kmemleak: Kernel memory leak detector disabled [ 0.000000] ftrace: allocating 41240 entries in 162 pages [ 0.000000] ftrace: allocated 162 pages with 3 groups [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4. [ 0.000000] rcu: RCU callback double-/use-after-free debug enabled. [ 0.000000] Rude variant of Tasks RCU enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 [ 0.000000] NR_IRQS: 524544, nr_irqs: 456, preallocated irqs: 16 [ 0.000000] random: get_random_bytes called from start_kernel+0x622/0x9a8 with crng_init=0 [ 0.001000] Console: colour *CGA 80x25 [ 0.001000] printk: console [ttyS1] enabled [ 0.001000] ACPI: Core revision 20220331 [ 0.001000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns [ 0.001015] APIC: Switch to symmetric I/O mode setup [ 0.003187] x2apic enabled [ 0.004008] Switched APIC routing to physical x2apic. [ 0.005010] kvm-guest: setup PV IPIs [ 0.008363] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 0.009000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 0.009043] Calibrating delay loop (skipped) preset value.. 4800.00 BogoMIPS (lpj=2400000) [ 0.010019] pid_max: default: 32768 minimum: 301 [ 0.011139] LSM: Security Framework initializing [ 0.013067] Yama: becoming mindful. [ 0.014053] SELinux: Initializing. [ 0.015087] *** VALIDATE selinux *** [ 0.025071] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, vmalloc) [ 0.029817] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, vmalloc) [ 0.030195] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.031129] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, vmalloc) [ 0.032156] *** VALIDATE tmpfs *** [ 0.034552] *** VALIDATE proc *** [ 0.036180] *** VALIDATE cgroup *** [ 0.037015] *** VALIDATE cgroup2 *** [ 0.039351] x86/cpu: User Mode Instruction Prevention (UMIP) activated [ 0.041122] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0 [ 0.042015] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0 [ 0.043044] Spectre V2 : User space: Vulnerable [ 0.044015] Speculative Store Bypass: Vulnerable [ 0.047825] debug: unmapping init [mem 0xffffffffb3c59000-0xffffffffb3c60fff] [ 0.050176] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz (family: 0x6, model: 0x3e, stepping: 0x4) [ 0.051844] Performance Events: IvyBridge events, full-width counters, Intel PMU driver. [ 0.052036] ... version: 2 [ 0.053021] ... bit width: 48 [ 0.054016] ... generic registers: 4 [ 0.055021] ... value mask: 0000ffffffffffff [ 0.056021] ... max period: 00007fffffffffff [ 0.057020] ... fixed-purpose events: 3 [ 0.058018] ... event mask: 000000070000000f [ 0.060290] rcu: Hierarchical SRCU implementation. [ 0.062795] smp: Bringing up secondary CPUs ... [ 0.063683] x86: Booting SMP configuration: [ 0.064032] .... node #0, CPUs: #1 #2 #3 [ 0.068235] smp: Brought up 1 node, 4 CPUs [ 0.070019] smpboot: Max logical packages: 1 [ 0.071017] smpboot: Total of 4 processors activated (19200.00 BogoMIPS) [ 0.259022] node 0 deferred pages initialised in 183ms [ 0.262555] devtmpfs: initialized [ 0.264297] x86/mm: Memory block size: 128MB [ 0.269125] gcov: version magic: 0x41383552 [ 0.272374] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns [ 0.276103] futex hash table entries: 1024 (order: 4, 65536 bytes, vmalloc) [ 0.279309] pinctrl core: initialized pinctrl subsystem [ 0.281298] [ 0.281867] ************************************************************* [ 0.285015] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.288012] ** ** [ 0.291028] ** IOMMU DebugFS SUPPORT HAS BEEN ENABLED IN THIS KERNEL ** [ 0.293010] ** ** [ 0.296027] ** This means that this kernel is built to expose internal ** [ 0.299095] ** IOMMU data structures, which may compromise security on ** [ 0.301023] ** your system. ** [ 0.304024] ** ** [ 0.307026] ** If you see this message and you are not debugging the ** [ 0.310017] ** kernel, report this immediately to your vendor! ** [ 0.312013] ** ** [ 0.314012] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE ** [ 0.317033] ************************************************************* [ 0.320736] NET: Registered protocol family 16 [ 0.322436] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations [ 0.326119] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations [ 0.329073] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations [ 0.333077] cpuidle: using governor menu [ 0.334728] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 [ 0.335563] PCI: Using configuration type 1 for base access [ 0.337138] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on [ 0.349060] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages [ 0.351072] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.355167] cryptd: max_cpu_qlen set to 1000 [ 0.359309] ACPI: Added _OSI(Module Device) [ 0.361023] ACPI: Added _OSI(Processor Device) [ 0.363019] ACPI: Added _OSI(3.0 _SCP Extensions) [ 0.364019] ACPI: Added _OSI(Processor Aggregator Device) [ 0.371367] ACPI: 1 ACPI AML tables successfully acquired and loaded [ 0.379573] ACPI: Interpreter enabled [ 0.380144] ACPI: PM: (supports S0 S3 S4 S5) [ 0.382020] ACPI: Using IOAPIC for interrupt routing [ 0.383146] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug [ 0.387472] ACPI: Enabled 2 GPEs in block 00 to 0F [ 0.398956] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) [ 0.401063] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI HPX-Type3] [ 0.404026] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI] [ 0.408120] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. [ 0.412762] acpiphp: Slot [2] registered [ 0.415197] acpiphp: Slot [5] registered [ 0.417191] acpiphp: Slot [6] registered [ 0.418236] acpiphp: Slot [7] registered [ 0.420238] acpiphp: Slot [8] registered [ 0.422180] acpiphp: Slot [9] registered [ 0.424117] acpiphp: Slot [10] registered [ 0.425159] acpiphp: Slot [3] registered [ 0.428129] acpiphp: Slot [4] registered [ 0.430102] acpiphp: Slot [11] registered [ 0.431088] acpiphp: Slot [12] registered [ 0.433095] acpiphp: Slot [13] registered [ 0.434288] acpiphp: Slot [14] registered [ 0.436130] acpiphp: Slot [15] registered [ 0.438101] acpiphp: Slot [16] registered [ 0.439145] acpiphp: Slot [17] registered [ 0.441086] acpiphp: Slot [18] registered [ 0.443091] acpiphp: Slot [19] registered [ 0.444097] acpiphp: Slot [20] registered [ 0.446104] acpiphp: Slot [21] registered [ 0.448285] acpiphp: Slot [22] registered [ 0.449083] acpiphp: Slot [23] registered [ 0.451089] acpiphp: Slot [24] registered [ 0.453187] acpiphp: Slot [25] registered [ 0.454095] acpiphp: Slot [26] registered [ 0.456163] acpiphp: Slot [27] registered [ 0.457094] acpiphp: Slot [28] registered [ 0.459091] acpiphp: Slot [29] registered [ 0.460082] acpiphp: Slot [30] registered [ 0.462113] acpiphp: Slot [31] registered [ 0.463106] PCI host bridge to bus 0000:00 [ 0.464021] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window] [ 0.466022] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window] [ 0.468024] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] [ 0.471044] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window] [ 0.473032] pci_bus 0000:00: root bus resource [mem 0xe0000000000-0xe007fffffff window] [ 0.474030] pci_bus 0000:00: root bus resource [bus 00-ff] [ 0.476215] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000 [ 0.479577] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100 [ 0.483307] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180 [ 0.493823] pci 0000:00:01.1: reg 0x20: [io 0xc320-0xc32f] [ 0.499109] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7] [ 0.502021] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6] [ 0.504022] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177] [ 0.506021] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376] [ 0.509770] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000 [ 0.512919] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI [ 0.516055] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB [ 0.520086] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 [ 0.525022] pci 0000:00:02.0: reg 0x10: [io 0xc300-0xc31f] [ 0.539033] pci 0000:00:02.0: reg 0x20: [mem 0xe0000000000-0xe0000003fff 64bit pref] [ 0.544020] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfebbffff pref] [ 0.550735] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 [ 0.564021] pci 0000:00:05.0: reg 0x10: [io 0xc000-0xc07f] [ 0.578035] pci 0000:00:05.0: reg 0x14: [mem 0xfebc0000-0xfebc0fff] [ 0.594020] pci 0000:00:05.0: reg 0x20: [mem 0xe0000004000-0xe0000007fff 64bit pref] [ 0.604000] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 [ 0.618023] pci 0000:00:06.0: reg 0x10: [io 0xc080-0xc0ff] [ 0.625021] pci 0000:00:06.0: reg 0x14: [mem 0xfebc1000-0xfebc1fff] [ 0.645027] pci 0000:00:06.0: reg 0x20: [mem 0xe0000008000-0xe000000bfff 64bit pref] [ 0.657625] pci 0000:00:07.0: [1af4:1001] type 00 class 0x010000 [ 0.665022] pci 0000:00:07.0: reg 0x10: [io 0xc100-0xc17f] [ 0.672024] pci 0000:00:07.0: reg 0x14: [mem 0xfebc2000-0xfebc2fff] [ 0.694022] pci 0000:00:07.0: reg 0x20: [mem 0xe000000c000-0xe000000ffff 64bit pref] [ 0.706073] pci 0000:00:08.0: [1af4:1001] type 00 class 0x010000 [ 0.714037] pci 0000:00:08.0: reg 0x10: [io 0xc180-0xc1ff] [ 0.723033] pci 0000:00:08.0: reg 0x14: [mem 0xfebc3000-0xfebc3fff] [ 0.740023] pci 0000:00:08.0: reg 0x20: [mem 0xe0000010000-0xe0000013fff 64bit pref] [ 0.755893] pci 0000:00:09.0: [1af4:1001] type 00 class 0x010000 [ 0.766023] pci 0000:00:09.0: reg 0x10: [io 0xc200-0xc27f] [ 0.779022] pci 0000:00:09.0: reg 0x14: [mem 0xfebc4000-0xfebc4fff] [ 0.800027] pci 0000:00:09.0: reg 0x20: [mem 0xe0000014000-0xe0000017fff 64bit pref] [ 0.813400] pci 0000:00:0a.0: [1af4:1001] type 00 class 0x010000 [ 0.828027] pci 0000:00:0a.0: reg 0x10: [io 0xc280-0xc2ff] [ 0.841025] pci 0000:00:0a.0: reg 0x14: [mem 0xfebc5000-0xfebc5fff] [ 0.872026] pci 0000:00:0a.0: reg 0x20: [mem 0xe0000018000-0xe000001bfff 64bit pref] [ 0.883823] ACPI: PCI: Interrupt link LNKA configured for IRQ 10 [ 0.888465] ACPI: PCI: Interrupt link LNKB configured for IRQ 10 [ 0.891534] ACPI: PCI: Interrupt link LNKC configured for IRQ 11 [ 0.894638] ACPI: PCI: Interrupt link LNKD configured for IRQ 11 [ 0.897404] ACPI: PCI: Interrupt link LNKS configured for IRQ 9 [ 0.903137] iommu: Default domain type: Passthrough [ 0.905433] SCSI subsystem initialized [ 0.907167] ACPI: bus type USB registered [ 0.908128] usbcore: registered new interface driver usbfs [ 0.910115] usbcore: registered new interface driver hub [ 0.912106] usbcore: registered new device driver usb [ 0.914201] pps_core: LinuxPPS API ver. 1 registered [ 0.916017] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 0.920070] PTP clock support registered [ 0.922180] EDAC MC: Ver: 3.0.0 [ 0.924150] PCI: Using ACPI for IRQ routing [ 0.927072] NetLabel: Initializing [ 0.928014] NetLabel: domain hash size = 128 [ 0.930014] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO [ 0.932112] NetLabel: unlabeled traffic allowed by default [ 0.934403] vgaarb: loaded [ 0.935301] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0 [ 0.937019] hpet0: 3 comparators, 64-bit 100.000000 MHz counter [ 0.945226] clocksource: Switched to clocksource kvm-clock [ 1.065328] VFS: Disk quotas dquot_6.6.0 [ 1.066848] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 1.070195] *** VALIDATE ramfs *** [ 1.071800] *** VALIDATE hugetlbfs *** [ 1.073614] pnp: PnP ACPI init [ 1.076584] pnp: PnP ACPI: found 6 devices [ 1.096575] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns [ 1.100156] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window] [ 1.102638] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window] [ 1.104650] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window] [ 1.107147] pci_bus 0000:00: resource 7 [mem 0xc0000000-0xfebfffff window] [ 1.109596] pci_bus 0000:00: resource 8 [mem 0xe0000000000-0xe007fffffff window] [ 1.112441] NET: Registered protocol family 2 [ 1.115360] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 1.120670] tcp_listen_portaddr_hash hash table entries: 4096 (order: 5, 163840 bytes, vmalloc) [ 1.123820] TCP established hash table entries: 65536 (order: 7, 524288 bytes, vmalloc) [ 1.129529] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, vmalloc) [ 1.133200] TCP: Hash tables configured (established 65536 bind 65536) [ 1.136564] MPTCP token hash table entries: 8192 (order: 6, 393216 bytes, vmalloc) [ 1.139339] UDP hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.142645] UDP-Lite hash table entries: 4096 (order: 6, 393216 bytes, vmalloc) [ 1.145972] NET: Registered protocol family 1 [ 1.148637] RPC: Registered named UNIX socket transport module. [ 1.150116] RPC: Registered udp transport module. [ 1.151269] RPC: Registered tcp transport module. [ 1.152370] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 1.154967] NET: Registered protocol family 44 [ 1.156770] pci 0000:00:00.0: Limiting direct PCI/PCI transfers [ 1.159126] pci 0000:00:01.0: PIIX3: Enabling Passive Release [ 1.161273] pci 0000:00:01.0: Activating ISA DMA hang workarounds [ 1.163786] PCI: CLS 0 bytes, default 64 [ 1.165523] Unpacking initramfs... [ 2.643330] debug: unmapping init [mem 0xffff8885bcc54000-0xffff8885bffbffff] [ 2.647026] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) [ 2.649202] software IO TLB: mapped [mem 0x00000000a8000000-0x00000000ac000000] (64MB) [ 2.652076] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x22983777dd9, max_idle_ns: 440795300422 ns [ 3.175546] Initialise system trusted keyrings [ 3.176693] Key type blacklist registered [ 3.177810] workingset: timestamp_bits=36 max_order=20 bucket_order=0 [ 3.185436] zbud: loaded [ 3.188351] *** VALIDATE nfs *** [ 3.189614] *** VALIDATE nfs4 *** [ 3.191097] pstore: using deflate compression [ 3.195350] Platform Keyring initialized [ 3.319596] NET: Registered protocol family 38 [ 3.320820] Key type asymmetric registered [ 3.321843] Asymmetric key parser 'x509' registered [ 3.323224] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247) [ 3.325579] io scheduler mq-deadline registered [ 3.327984] io scheduler kyber registered [ 3.330952] io scheduler bfq registered [ 3.332281] atomic64_test: passed for x86-64 platform with CX8 and with SSE [ 3.334386] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 [ 3.336420] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0 [ 3.338869] ACPI: Power Button [PWRF] [ 3.343696] ACPI: \_SB_.LNKB: Enabled at IRQ 10 [ 3.348702] ACPI: \_SB_.LNKA: Enabled at IRQ 11 [ 3.358267] ACPI: \_SB_.LNKC: Enabled at IRQ 11 [ 3.364800] ACPI: \_SB_.LNKD: Enabled at IRQ 10 [ 3.381693] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 3.411129] 00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A [ 3.439914] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A [ 3.445265] Non-volatile memory driver v1.3 [ 3.447197] Linux agpgart interface v0.103 [ 3.485294] virtio_blk virtio1: [vda] 145960 512-byte logical blocks (74.7 MB/71.3 MiB) [ 3.488545] vda: detected capacity change from 0 to 74731520 [ 3.502616] virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) [ 3.505538] vdb: detected capacity change from 0 to 1073741824 [ 3.522690] virtio_blk virtio3: [vdc] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.525754] vdc: detected capacity change from 0 to 2621440000 [ 3.545205] virtio_blk virtio4: [vdd] 5120000 512-byte logical blocks (2.62 GB/2.44 GiB) [ 3.548438] vdd: detected capacity change from 0 to 2621440000 [ 3.565124] virtio_blk virtio5: [vde] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.568395] vde: detected capacity change from 0 to 4294967296 [ 3.584758] virtio_blk virtio6: [vdf] 8388608 512-byte logical blocks (4.29 GB/4.00 GiB) [ 3.587959] vdf: detected capacity change from 0 to 4294967296 [ 3.596254] libphy: Fixed MDIO Bus: probed [ 3.602786] usbcore: registered new interface driver usbserial_generic [ 3.605433] usbserial: USB Serial support registered for generic [ 3.607869] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12 [ 3.612709] serio: i8042 KBD port at 0x60,0x64 irq 1 [ 3.614220] serio: i8042 AUX port at 0x60,0x64 irq 12 [ 3.616773] mousedev: PS/2 mouse device common for all mice [ 3.621614] rtc_cmos 00:05: RTC can wake from S4 [ 3.624518] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1 [ 3.625317] rtc_cmos 00:05: registered as rtc0 [ 3.629923] rtc_cmos 00:05: alarms up to one day, y3k, 242 bytes nvram, hpet irqs [ 3.632318] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4 [ 3.633945] intel_pstate: CPU model not supported [ 3.638033] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3 [ 3.642682] hid: raw HID events driver (C) Jiri Kosina [ 3.644860] usbcore: registered new interface driver usbhid [ 3.646937] usbhid: USB HID core driver [ 3.648690] drop_monitor: Initializing network drop monitor service [ 3.651387] Initializing XFRM netlink socket [ 3.653704] NET: Registered protocol family 10 [ 3.657235] Segment Routing with IPv6 [ 3.658785] NET: Registered protocol family 17 [ 3.660861] mpls_gso: MPLS GSO support [ 3.667190] RAS: Correctable Errors collector initialized. [ 3.669457] AVX version of gcm_enc/dec engaged. [ 3.671115] AES CTR mode by8 optimization enabled [ 3.765722] sched_clock: Marking stable (3765641490, 0)->(4716006810, -950365320) [ 3.770218] registered taskstats version 1 [ 3.771758] Loading compiled-in X.509 certificates [ 3.773532] zswap: loaded using pool lzo/zbud [ 3.811262] Key type big_key registered [ 3.823724] Key type encrypted registered [ 3.825910] ima: No TPM chip found, activating TPM-bypass! [ 3.831918] ima: Allocated hash algorithm: sha1 [ 3.833341] ima: No architecture policies found [ 3.834740] evm: Initialising EVM extended attributes: [ 3.836290] evm: security.selinux [ 3.837279] evm: security.ima [ 3.838415] evm: security.capability [ 3.839781] evm: HMAC attrs: 0x1 [ 3.842227] rtc_cmos 00:05: setting system clock to 2026-09-08 21:34:54 UTC (1788903294) [ 3.848796] debug: unmapping init [mem 0xffffffffb4c03000-0xffffffffb4dfffff] [ 3.851927] debug: unmapping init [mem 0xffffffffb3982000-0xffffffffb3c58fff] [ 3.860269] Write protecting the kernel read-only data: 28672k [ 3.864069] debug: unmapping init [mem 0xffffffffb2003000-0xffffffffb21fffff] [ 3.867777] debug: unmapping init [mem 0xffffffffb2914000-0xffffffffb29fffff] [ 3.911302] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 3.921727] systemd[1]: Detected virtualization kvm. [ 3.924118] systemd[1]: Detected architecture x86-64. [ 3.926309] systemd[1]: Running in initial RAM disk. Welcome to Rocky Linux 8.10 (Green Obsidian) dracut-049-233.git20240115.el8 (Initramfs)! [ 3.954926] systemd[1]: No hostname configured. [ 3.957127] systemd[1]: Set hostname to . [ 3.959515] random: systemd: uninitialized urandom read (16 bytes read) [ 3.962172] systemd[1]: Initializing machine ID from random generator. [ 4.045588] random: ln: uninitialized urandom read (6 bytes read) [ 4.151537] random: systemd: uninitialized urandom read (16 bytes read) [ 4.155437] systemd[1]: Listening on Journal Socket. [ OK ] Listening on Journal Socket. [ 4.160754] systemd[1]: Listening on udev Kernel Socket. [ OK ] Listening on udev Kernel Socket. [ 4.165187] systemd[1]: Reached target Slices. [ OK ] Reached target Slices. [ OK ] Reached target Timers. [ OK ] Reached target Initrd Root Device. [ OK ] Reached target Local File Systems. [ OK ] Listening on Journal Socket (/dev/log). Starting Apply Kernel Variables... Starting Setup Virtual Console... Starting Create Volatile Files and Directories... [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Sockets. [ OK ] Started Memstrack Anylazing Service. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. Starting Create list of required st…ce nodes for the current kernel... Starting Journal Service... [ OK ] Started Apply Kernel Variables. [ OK ] Started Setup Virtual Console. [ OK ] Started Create Volatile Files and Directories. [ OK ] Started Create list of required sta…vice nodes for the current kernel. Starting Create Static Device Nodes in /dev... Starting dracut cmdline hook... [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Started Journal Service. [ OK ] Started dracut cmdline hook. Starting dracut pre-udev hook... [ 4.832980] device-mapper: uevent: version 1.0.3 [ 4.834985] device-mapper: ioctl: 4.46.0-ioctl (2022-02-22) initialised: dm-devel@redhat.com [ OK ] Started dracut pre-udev hook. Starting udev Kernel Device Manager... [ OK ] Started udev Kernel Device Manager. Starting dracut pre-trigger hook... [ OK ] Started dracut pre-trigger hook. Starting udev Coldplug all Devices... Mounting Kernel Configuration File System... [ OK ] Mounted Kernel Configuration File System. [ OK ] Started udev Coldplug all Devices. [ OK ] Reached target System Initialization. [ OK ] Reached target Basic System. [ OK ] Started Hardware RNG Entropy Gatherer Daemon. Starting dracut initqueue hook... [ 5.717391] random: fast init done [ 5.738800] virtio_net virtio0 ens2: renamed from eth0 [ 5.926641] scsi host0: ata_piix [ 5.949699] scsi host1: ata_piix [ 5.953981] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc320 irq 14 [ 5.956751] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc328 irq 15 [ 9.620233] dracut-initqueue[587]: RTNETLINK answers: File exists [ 10.344924] random: crng init done [ 10.348283] random: 7 urandom warning(s) missed due to ratelimiting Starting nbd nbd0... [ OK ] Started nbd nbd0. [ OK ] Started dracut initqueue hook. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Mounting /sysroot... [ 11.590614] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null) [ OK ] Mounted /sysroot. [ OK ] Reached target Initrd Root File System. Starting Reload Configuration from the Real Root... [ OK ] Started Reload Configuration from the Real Root. [ OK ] Reached target Initrd File Systems. [ OK ] Reached target Initrd Default Target. Starting dracut pre-pivot and cleanup hook... [ OK ] Started dracut pre-pivot and cleanup hook. Starting Cleaning Up and Shutting Down Daemons... [ OK ] Stopped target Timers. [ OK ] Stopped dracut pre-pivot and cleanup hook. [ OK ] Stopped target Initrd Default Target. [ OK ] Stopped target Initrd Root Device. [ OK ] Stopped target Remote File Systems. [ OK ] Stopped target Remote File Systems (Pre). [ OK ] Stopped dracut initqueue hook. Stopping Hardware RNG Entropy Gatherer Daemon... [ OK ] Stopped Hardware RNG Entropy Gatherer Daemon. [ OK ] Started Cleaning Up and Shutting Down Daemons. [ OK ] Stopped target Basic System. [ OK ] Stopped target Sockets. [ OK ] Stopped target System Initialization. [ OK ] Stopped udev Coldplug all Devices. [ OK ] Stopped dracut pre-trigger hook. [ OK ] Stopped Apply Kernel Variables. [ OK ] Stopped target Local Encrypted Volumes. [ OK ] Stopped Create Volatile Files and Directories. [ OK ] Stopped target Local File Systems. Stopping udev Kernel Device Manager... [ OK ] Stopped target Swap. [ OK ] Stopped target Slices. [ OK ] Stopped target Paths. [ OK ] Stopped Dispatch Password Requests to Console Directory Watch. [ OK ] Stopped udev Kernel Device Manager. [ OK ] Stopped dracut pre-udev hook. [ OK ] Stopped dracut cmdline hook. [ OK ] Stopped Create Static Device Nodes in /dev. [ OK ] Stopped Create list of required sta…vice nodes for the current kernel. [ OK ] Closed udev Kernel Socket. [ OK ] Closed udev Control Socket. Starting Cleanup udevd DB... [ OK ] Started Cleanup udevd DB. [ OK ] Reached target Switch Root. Starting Switch Root... [ 12.976547] printk: systemd: 26 output lines suppressed due to ratelimiting [ 13.248995] SELinux: Disabled at runtime. [ 13.314558] systemd[1]: systemd 239 (239-82.el8_10.5) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy) [ 13.322482] systemd[1]: Detected virtualization kvm. [ 13.324579] systemd[1]: Detected architecture x86-64. Welcome to Rocky Linux 8.10 (Green Obsidian)! [ 13.810395] systemd[1]: initrd-switch-root.service: Succeeded. [ 13.812890] systemd[1]: Stopped Switch Root. [ OK ] Stopped Switch Root. [ 13.817534] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart. [ 13.821383] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. [ 13.824120] systemd[1]: Stopped Journal Service. [ OK ] Stopped Journal Service. [ 13.834704] systemd[1]: Starting Journal Service... Starting Journal Service... [ 13.846440] systemd[1]: Created slice system-getty.slice. [ OK ] Created slice system-getty.slice. [ OK ] Listening on udev Control Socket. Mounting Kernel Debug File System... Mounting POSIX Message Queue File System... Mounting Huge Pages File System... Starting Apply Kernel Variables... [ OK ] Listening on udev Kernel Socket. [ OK ] Created slice User and Session Slice. [ OK ] Stopped target Switch Root. Starting Create list of required st…ce nodes for the current kernel... Starting udev Coldplug all Devices... [ OK ] Reached target Slices. [ OK ] Stopped target Initrd File Systems. [ OK ] Created slice system-sshd\x2dkeygen.slice. [FAILED] Failed to set up automount Arbitrar…rmats File System Automount Point. See 'systemctl status proc-sys-fs-binfmt_misc.automount' for details. [ OK ] Reached target rpc_pipefs.target. [ OK ] Stopped target Initrd Root File System. [ OK ] Listening on initctl Compatibility Named Pipe. Activating swap /dev/disk/by-label/SWAP... [ OK ] Created slice system-serial\x2dgetty.slice. Starting Remount Root and Kernel File Systems... [ 13.999639] Adding 1048572k swap on /dev/vdb. Priority:-2 extents:1 across:1048572k FS [ OK ] Listening on Process Core Dump Socket. [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on RPCbind Server Activation Socket. [ OK ] Reached target RPC Port Mapper. [ OK ] Started Dispatch Password Requests to Console Directory Watch. [ OK ] Reached target Paths. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Started Journal Service. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Mounted Huge Pages File System. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create list of required sta…vice nodes for the current kernel. [ OK ] Activated swap /dev/disk/by-label/SWAP. [FAILED] Failed to start Remount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. Starting Configure read-only root support... [ OK ] Reached target Swap. Starting Create Static Device Nodes in /dev... Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /home/green/git/lustre-release... Mounting /mnt... Starting udev Kernel Device Manager... [ OK ] Mounted /mnt. [ 14.315923] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ OK ] Started udev Coldplug all Devices. [ OK ] Mounted /home/green/git/lustre-release. [ OK ] Started udev Kernel Device Manager. [ 14.672316] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0 [ 14.768373] input: PC Speaker as /devices/platform/pcspkr/input/input5 [ 14.887122] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer [ 14.896688] EDAC sbridge: Ver: 1.1.2 [ 16.501676] Key type dns_resolver registered [ 16.817502] NFS: Registering the id_resolver key type [ 16.819831] Key type id_resolver registered [ 16.821750] Key type id_legacy registered [ OK ] Started Configure read-only root support. [ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Rebuild Dynamic Linker Cache... Starting Load/Save Random Seed... Starting Create Volatile Files and Directories... [ OK ] Started Mark the need to relabel after reboot. [ OK ] Started Load/Save Random Seed. [ OK ] Started Create Volatile Files and Directories. Starting RPC Bind... Starting Update UTMP about System Boot/Shutdown... [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started RPC Bind. [ OK ] Started Rebuild Dynamic Linker Cache. Starting Update is Completed... [ OK ] Started Update is Completed. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started dnf makecache --timer. [ OK ] Started daily update of the root trust anchor for DNSSEC. [ OK ] Reached target Timers. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. [ OK ] Started D-Bus System Message Bus. Starting Restore /run/initramfs on shutdown... [ OK ] Started Hardware RNG Entropy Gatherer Daemon. [ OK ] Reached target sshd-keygen.target. [ OK ] Started irqbalance daemon. Starting Network Manager... Starting Login Service... [ OK ] Started Restore /run/initramfs on shutdown. [ OK ] Started Network Manager. Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting GSSAPI Proxy Daemon... Starting OpenSSH server daemon... Starting Dynamic System Tuning Daemon... [ OK ] Started Login Service. Starting Hostname Service... [ OK ] Started GSSAPI Proxy Daemon. [ OK ] Reached target NFS client services. [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target Remote File Systems. Starting Permit User Sessions... [ OK ] Started OpenSSH server daemon. [ OK ] Started Permit User Sessions. [ OK ] Started Getty on tty1. [ OK ] Started Serial Getty on ttyS1. [ OK ] Started Serial Getty on ttyS0. [ OK ] Reached target Login Prompts. [ OK ] Started Command Scheduler. [ OK ] Started Hostname Service. Starting Network Manager Script Dispatcher Service... [ OK ] Started Network Manager Script Dispatcher Service. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting Crash recovery kernel arming... Starting System Logging Service... Starting Notify NFS peers of a restart... [ OK ] Started Notify NFS peers of a restart. [ OK ] Started System Logging Service. Starting Authorization Manager... [ OK ] Started Authorization Manager. [ OK ] Started Dynamic System Tuning Daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Rocky Linux 8.10 (Green Obsidian) Kernel 4.18.0rh8.10-debug on an x86_64 oleg106-server login: [ 46.125441] hrtimer: interrupt took 3708440 ns [ 67.314692] libcfs: loading out-of-tree module taints kernel. [ 67.422737] Key type ._llcrypt registered [ 67.437072] Key type .llcrypt registered [ 67.629843] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 88.049509] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 89.316878] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 89.326567] alg: No test for adler32 (adler32-zlib) [ 90.915575] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 92.121083] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 93.896286] Key type lgssc registered [ 95.828584] Lustre: Echo OBD driver; http://www.lustre.org/ [ 114.906024] ZFS: Loaded module v2.3.2-1, ZFS pool version 5000, ZFS filesystem version 5 [ 152.359266] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 162.713464] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 162.757730] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 163.923466] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 163.942409] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 164.040861] Lustre: lustre-MDT0000: new disk, initializing [ 164.180487] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 164.197674] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 167.793398] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 178.967493] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 179.035308] Lustre: 6516:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 179.100913] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 179.105337] Lustre: Skipped 1 previous similar message [ 179.198976] Lustre: lustre-MDT0001: new disk, initializing [ 179.248446] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 179.279954] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 179.287736] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 183.312532] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 187.194411] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 194.738138] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 194.904770] Lustre: lustre-OST0000: new disk, initializing [ 194.909716] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 194.915578] Lustre: 8451:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 194.966230] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 197.079827] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 197.095050] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 197.141278] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 198.991090] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 208.591300] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 208.675521] Lustre: lustre-OST0001: new disk, initializing [ 208.678433] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 208.685956] Lustre: 9524:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 208.722883] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 211.635374] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug vfstrace rpctrace dlmtrace neterror ha config ioctl super lfsck all [ 215.580603] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 215.587768] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 215.614442] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 219.323805] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 224.015586] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 226.696570] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing check_logdir /tmp/testlogs/ [ 229.001583] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing yml_node [ 231.572568] Lustre: DEBUG MARKER: Client: 2.17.55.26 [ 232.955652] Lustre: DEBUG MARKER: MDS: 2.17.55.26 [ 234.369207] Lustre: DEBUG MARKER: OSS: 2.17.55.26 [ 235.287632] Lustre: DEBUG MARKER: -----============= acceptance-small: conf-sanity ============----- Tue Sep 8 17:38:44 EDT 2026 [ 245.155879] Lustre: DEBUG MARKER: excepting tests: 102 106 115 32newtarball [ 245.917278] Lustre: DEBUG MARKER: skipping tests SLOW=no: 45 69 106 111 114 [ 261.088912] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 261.093443] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 261.098675] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 261.602122] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 261.603841] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 261.608899] Lustre: Skipped 2 previous similar messages [ 261.610859] Lustre: Skipped 1 previous similar message [ 264.441705] Lustre: server umount lustre-MDT0000 complete [ 266.137692] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903556 with bad export cookie 3660647728251634417 [ 266.138330] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 266.149959] LustreError: 6508:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 266.721356] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 266.721880] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 266.726267] Lustre: Skipped 1 previous similar message [ 266.731061] Lustre: Skipped 2 previous similar messages [ 271.841442] LustreError: 6523:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 271.841846] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 271.848621] LustreError: 6523:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 271.855753] Lustre: Skipped 1 previous similar message [ 272.389561] Lustre: server umount lustre-MDT0001 complete [ 274.369496] Lustre: server umount lustre-OST0000 complete [ 276.297755] Lustre: server umount lustre-OST0001 complete [ 279.407318] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 283.565247] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 300.503882] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 300.656645] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 300.677865] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 300.717891] Lustre: lustre-MDT0000: new disk, initializing [ 300.770080] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 300.781465] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 302.633674] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 307.904160] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 307.967280] Lustre: 15064:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 308.113885] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 309.788760] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 314.570510] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 317.222731] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 320.318835] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 320.454141] Lustre: lustre-OST0000: new disk, initializing [ 320.456482] Lustre: Skipped 1 previous similar message [ 320.459058] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 320.462127] Lustre: Skipped 2 previous similar messages [ 320.465648] Lustre: 17028:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 320.505258] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 320.507878] Lustre: Skipped 1 previous similar message [ 322.374066] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 322.377870] Lustre: Skipped 1 previous similar message [ 322.380645] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 322.437479] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 323.010213] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 327.430595] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 329.028598] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 329.102829] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 330.723868] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 330.803343] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 332.770557] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 332.771499] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 332.773843] LustreError: Skipped 1 previous similar message [ 332.780439] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 332.783466] Lustre: Skipped 1 previous similar message [ 337.694985] Lustre: server umount lustre-OST0000 complete [ 343.009500] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 343.010306] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 343.014640] Lustre: Skipped 2 previous similar messages [ 345.308199] Lustre: server umount lustre-MDT0000 complete [ 346.518328] LustreError: 15056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903637 with bad export cookie 3660647728251635950 [ 346.521788] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 346.522467] LustreError: 15056:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 346.632262] Lustre: server umount lustre-MDT0001 complete [ 348.324406] Lustre: DEBUG MARKER: == conf-sanity test 76a: set permanent params with lctl across mounts ========================================================== 17:40:38 (1788903638) [ 350.608534] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 350.802660] LustreError: 19292:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 350.829990] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 352.343904] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 355.828574] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 357.495854] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 359.932500] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 361.909949] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 364.369363] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 366.524971] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 369.233176] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 381.590233] Lustre: Modifying parameter general.osc.*.max_dirty_mb=452 in log params [ 386.018795] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 386.020358] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 386.022750] Lustre: Skipped 2 previous similar messages [ 386.026092] Lustre: Skipped 3 previous similar messages [ 390.951273] Lustre: server umount lustre-MDT0000 complete [ 391.138128] LustreError: 19292:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 391.143338] LustreError: 19292:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 392.266299] LustreError: 19272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903682 with bad export cookie 3660647728251636832 [ 392.268211] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 392.269740] LustreError: 19272:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 392.388652] Lustre: server umount lustre-MDT0001 complete [ 404.126637] Lustre: server umount lustre-OST0000 complete [ 411.473476] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 415.605703] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 415.779820] LustreError: 23687:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 415.803216] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 415.805500] Lustre: Skipped 2 previous similar messages [ 417.478801] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 420.833400] LustreError: 23688:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 421.227228] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 423.096294] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 424.226818] Lustre: Setting parameter general.debug_raw_pointers=Y in log params [ 426.793144] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 429.055637] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 432.442847] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 432.514143] Lustre: lustre-OST0001: new disk, initializing [ 432.516531] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 432.519559] Lustre: 26002:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 434.120136] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 434.124476] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 434.166820] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 434.724136] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 439.449273] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 440.710387] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 446.436093] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 446.438067] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 446.441071] Lustre: Skipped 3 previous similar messages [ 446.443250] Lustre: Skipped 2 previous similar messages [ 450.016610] Lustre: server umount lustre-MDT0000 complete [ 451.488328] LustreError: 23669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903742 with bad export cookie 3660647728251637679 [ 451.491903] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 451.494235] LustreError: 23669:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 451.553483] LustreError: 26589:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 451.559132] LustreError: 26589:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 451.625733] Lustre: server umount lustre-MDT0001 complete [ 459.128481] Lustre: server umount lustre-OST0000 complete [ 460.361276] Lustre: server umount lustre-OST0001 complete [ 465.850421] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 469.478286] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 469.621561] LustreError: 28961:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 471.114103] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 474.165195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 475.806249] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 476.903798] Lustre: 30107:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 479.225135] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 481.500880] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 484.850416] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 484.945530] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 484.947938] Lustre: Skipped 6 previous similar messages [ 486.968958] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 490.142641] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 491.449533] Lustre: 31988:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 495.075270] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 495.076291] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 495.079499] Lustre: Skipped 5 previous similar messages [ 499.225832] Lustre: server umount lustre-MDT0000 complete [ 500.479257] LustreError: 28942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903791 with bad export cookie 3660647728251638911 [ 500.483786] LustreError: 28942:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 500.484051] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 500.600386] Lustre: server umount lustre-MDT0001 complete [ 511.490075] Lustre: server umount lustre-OST0000 complete [ 523.109529] Lustre: server umount lustre-OST0001 complete [ 526.287108] Lustre: DEBUG MARKER: == conf-sanity test 76b: verify params log setup correctly ========================================================== 17:43:36 (1788903816) [ 536.415739] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 539.808255] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 539.934267] LustreError: 34781:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 539.941581] LustreError: 34781:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 7 previous similar messages [ 541.373397] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 544.446766] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 545.938675] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 546.904970] Lustre: 35930:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 548.925524] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 550.787431] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 553.592039] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 555.462139] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 558.481034] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 559.684748] Lustre: 37808:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 565.220044] Lustre: lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 565.221043] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 565.222582] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 565.222589] Lustre: Skipped 7 previous similar messages [ 565.225249] Lustre: Skipped 3 previous similar messages [ 567.065182] Lustre: server umount lustre-MDT0000 complete [ 568.303189] LustreError: 37810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903858 with bad export cookie 3660647728251639996 [ 568.305934] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 568.306803] LustreError: 37810:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 568.420394] Lustre: server umount lustre-MDT0001 complete [ 579.521303] Lustre: server umount lustre-OST0000 complete [ 590.999878] Lustre: server umount lustre-OST0001 complete [ 593.785529] Lustre: DEBUG MARKER: == conf-sanity test 76c: verify changelog_mask is applied with lctl set_param -P ========================================================== 17:44:43 (1788903883) [ 598.889615] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 602.540392] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 604.019113] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 606.807569] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 608.273980] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 609.202495] Lustre: 41091:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 611.376430] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 612.516898] LustreError: 41446:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 612.523926] LustreError: 41446:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 613.368000] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 616.283988] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 616.369739] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 616.372760] Lustre: Skipped 7 previous similar messages [ 618.307368] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 621.416689] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 622.839751] Lustre: Setting parameter general.mdd.*.changelog_mask=-CLOSE in log params [ 622.841614] Lustre: Skipped 1 previous similar message [ 625.944456] Lustre: server umount lustre-MDT0000 complete [ 627.298965] LustreError: 41094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903917 with bad export cookie 3660647728251641081 [ 627.302469] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 627.303510] LustreError: 41094:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 633.454714] Lustre: server umount lustre-MDT0001 complete [ 636.260974] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 636.896920] LustreError: 3648:0:(client.c:1391:ptlrpc_import_delay_req()) @@@ invalidate in flight req@ffff888502416a00 x1875801152977280/t0(0) o250->MGC192.168.201.106@tcp@0@lo:26/25 lens 520/544 e 0 to 0 dl 0 ref 1 fl Rpc:NQU/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 638.433740] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 641.120200] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 642.536409] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 644.791307] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 655.586225] Lustre: DEBUG MARKER: mdc.lustre-MDT0000-mdc-*.mds_server_uuid in FULL state after 10 sec [ 656.872705] Lustre: lustre-MDT0000-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 657.219988] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 657.724120] Lustre: DEBUG MARKER: mdc.lustre-MDT0001-mdc-*.mds_server_uuid in FULL state after 0 sec [ 661.989308] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 665.045533] Lustre: server umount lustre-MDT0000 complete [ 666.192467] LustreError: 40693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788903956 with bad export cookie 3660647728251642159 [ 666.195843] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 666.196849] LustreError: 40693:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 666.302517] Lustre: server umount lustre-MDT0001 complete [ 677.766240] Lustre: server umount lustre-OST0000 complete [ 689.244875] Lustre: server umount lustre-OST0001 complete [ 691.992768] Lustre: DEBUG MARKER: == conf-sanity test 76d: verify llite.*.xattr_cache can be set by 'lctl set_param -P' correctly ========================================================== 17:46:21 (1788903981) [ 696.809977] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 699.964389] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 701.319572] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 703.908133] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 705.343081] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 706.263459] Lustre: 48734:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 706.266650] Lustre: 48734:0:(mgs_llog.c:1450:mgs_modify_param()) Skipped 1 previous similar message [ 708.308633] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 710.145176] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 712.702881] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 714.573113] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 717.469463] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 718.781123] Lustre: Setting parameter general.llite.*.xattr_cache=0 in log params [ 724.451739] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 724.454648] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 724.456469] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 724.457568] Lustre: Skipped 10 previous similar messages [ 724.462190] Lustre: Skipped 16 previous similar messages [ 729.045221] Lustre: server umount lustre-MDT0000 complete [ 730.241602] Lustre: server umount lustre-MDT0001 complete [ 741.898926] Lustre: server umount lustre-OST0000 complete [ 752.478945] Lustre: server umount lustre-OST0001 complete [ 755.351931] Lustre: DEBUG MARKER: == conf-sanity test 77: comma-separated MGS NIDs and failover node NIDs ========================================================== 17:47:25 (1788904045) [ 756.094303] Lustre: DEBUG MARKER: SKIP: conf-sanity test_77 mixed loopback and real device not working [ 756.602888] Lustre: DEBUG MARKER: == conf-sanity test 78: run resize2fs on MDT and OST filesystems ========================================================== 17:47:26 (1788904046) [ 759.689708] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 762.129718] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 769.040353] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 769.125458] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 769.157171] Lustre: lustre-MDT0000: new disk, initializing [ 769.179088] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 770.419648] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 773.990339] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 776.098044] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 776.198690] Lustre: 54803:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 777.597804] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 777.607395] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 778.028682] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 781.556887] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 782.226780] Lustre: DEBUG MARKER: create test files [ 793.057482] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 797.191856] Lustre: server umount lustre-OST0000 complete [ 798.633115] Lustre: server umount lustre-MDT0000 complete [ 802.455871] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 803.541784] Key type lgssc unregistered [ 803.687059] LNet: 56405:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 803.691438] LNetError: 56405:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 803.701368] LNet: Removed LNI 192.168.201.106@tcp [ 804.069098] Key type .llcrypt unregistered [ 804.070184] Key type ._llcrypt unregistered [ 814.109904] Key type ._llcrypt registered [ 814.111081] Key type .llcrypt registered [ 814.147323] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 814.546758] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 814.553332] alg: No test for adler32 (adler32-zlib) [ 815.401983] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 815.491480] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 817.064111] Key type lgssc registered [ 817.387272] Lustre: Echo OBD driver; http://www.lustre.org/ [ 819.830725] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 819.841960] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 820.974835] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 822.198535] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 824.366310] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 826.481676] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 826.577975] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 828.297508] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 830.600781] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 830.692601] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:102 to 0x240000400:129) [ 830.719268] Lustre: 59113:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 511 < left 544, rollback = 9 [ 830.722060] Lustre: 59113:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/4, destroy: 0/0/0 [ 830.724448] Lustre: 59113:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 830.727583] Lustre: 59113:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 830.730178] Lustre: 59113:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 830.732442] Lustre: 59113:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 833.553628] Lustre: DEBUG MARKER: check files after expanding the MDT and OST filesystems [ 834.988724] Lustre: DEBUG MARKER: create more files after expanding the MDT and OST filesystems [ 841.185553] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 841.185779] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 841.192166] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 842.828613] Lustre: server umount lustre-OST0000 complete [ 844.029222] Lustre: server umount lustre-MDT0000 complete [ 847.634807] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 848.705149] Key type lgssc unregistered [ 848.843798] LNet: 60718:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 848.846715] LNetError: 60718:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 848.852271] LNet: Removed LNI 192.168.201.106@tcp [ 849.169743] Key type .llcrypt unregistered [ 849.170782] Key type ._llcrypt unregistered [ 859.763884] Key type ._llcrypt registered [ 859.764840] Key type .llcrypt registered [ 859.795364] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 860.098540] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 860.147599] alg: No test for adler32 (adler32-zlib) [ 860.989293] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 861.062928] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 862.632157] Key type lgssc registered [ 863.056741] Lustre: Echo OBD driver; http://www.lustre.org/ [ 865.616066] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 865.622474] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 866.775968] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 868.510260] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 873.086373] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 878.782166] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 879.513911] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 887.260390] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 891.776366] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x240000400:140 to 0x240000400:161) [ 891.876917] Lustre: 63525:0:(osd_internal.h:1461:osd_trans_exec_op()) lustre-OST0000: opcode 8: before 509 < left 544, rollback = 9 [ 891.893568] Lustre: 63525:0:(osd_handler.c:2090:osd_trans_dump_creds()) create: 32/128/6, destroy: 0/0/0 [ 891.914130] Lustre: 63525:0:(osd_handler.c:2097:osd_trans_dump_creds()) attr_set: 0/0/0, xattr_set: 0/0/0 [ 891.921792] Lustre: 63525:0:(osd_handler.c:2107:osd_trans_dump_creds()) write: 2/18/1, punch: 0/0/0, quota 1/3/0 [ 891.929705] Lustre: 63525:0:(osd_handler.c:2114:osd_trans_dump_creds()) insert: 32/544/0, delete: 0/0/0 [ 891.943760] Lustre: 63525:0:(osd_handler.c:2121:osd_trans_dump_creds()) ref_add: 0/0/0, ref_del: 0/0/0 [ 893.986059] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 899.539386] Lustre: DEBUG MARKER: check files after shrinking the MDT and OST filesystems [ 912.354871] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 912.358451] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 912.363681] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 917.475490] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 918.687465] Lustre: server umount lustre-OST0000 complete [ 922.875117] Lustre: server umount lustre-MDT0000 complete [ 932.718780] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 936.548045] Key type lgssc unregistered [ 937.317165] LNet: 65107:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 937.332570] LNetError: 65107:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 937.376698] LNet: Removed LNI 192.168.201.106@tcp [ 939.243051] Key type .llcrypt unregistered [ 939.244540] Key type ._llcrypt unregistered [ 976.932949] Key type ._llcrypt registered [ 976.940940] Key type .llcrypt registered [ 977.183624] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 1000.846734] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1002.331836] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1002.630708] alg: No test for adler32 (adler32-zlib) [ 1003.842696] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 1004.248246] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 1006.010489] Key type lgssc registered [ 1007.449569] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1043.904196] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1043.959770] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1045.238755] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1045.279720] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1045.397917] Lustre: lustre-MDT0000: new disk, initializing [ 1045.478610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1045.494627] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1050.622325] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1061.874106] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1061.988818] Lustre: 68265:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1062.045271] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1062.048852] Lustre: Skipped 1 previous similar message [ 1062.117674] Lustre: lustre-MDT0001: new disk, initializing [ 1062.275010] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1062.337740] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1062.356480] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1066.690438] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1076.268031] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1081.764241] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1088.513661] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1088.706199] Lustre: lustre-OST0000: new disk, initializing [ 1088.712915] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1088.717914] Lustre: 70233:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1088.805889] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1094.888080] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1098.358585] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1098.384144] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1098.591879] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1104.780765] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1109.457301] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1109.699582] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1114.013750] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1114.301891] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1119.201420] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1119.204646] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1119.212623] LustreError: Skipped 1 previous similar message [ 1119.222897] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1119.232364] Lustre: Skipped 1 previous similar message [ 1119.259945] Lustre: Skipped 1 previous similar message [ 1122.797383] Lustre: server umount lustre-OST0000 complete [ 1128.929409] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1128.945544] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1128.960451] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1132.477719] Lustre: server umount lustre-MDT0000 complete [ 1136.001449] LustreError: 71775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788904426 with bad export cookie 10133850152573114554 [ 1136.006163] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1136.012672] LustreError: 71775:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1136.324979] Lustre: server umount lustre-MDT0001 complete [ 1144.866571] Lustre: DEBUG MARKER: == conf-sanity test 79: format MDT/OST without mgs option (should return errors) ========================================================== 17:53:53 (1788904433) [ 1155.864776] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1192.895747] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 1202.862530] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1241.694151] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1241.903852] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1241.925609] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1242.016862] Lustre: lustre-MDT0000: new disk, initializing [ 1242.166639] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1242.194430] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1247.528279] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1258.352306] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1258.490353] Lustre: 76255:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1258.566205] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 1258.578806] Lustre: Skipped 1 previous similar message [ 1258.705358] Lustre: lustre-MDT0001: new disk, initializing [ 1258.837692] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 1258.875379] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 1258.881974] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1264.310982] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1275.208710] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1280.761955] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1287.134845] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1287.399447] Lustre: lustre-OST0000: new disk, initializing [ 1287.403360] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 1287.413026] Lustre: 78223:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1287.528360] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1294.087849] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1296.996941] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 1297.013222] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1297.169801] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1303.239185] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1307.560875] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1307.781334] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1311.628897] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1311.836607] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1317.863311] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1317.864984] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1317.870596] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1317.870605] Lustre: Skipped 2 previous similar messages [ 1317.890706] Lustre: Skipped 2 previous similar messages [ 1317.912451] LustreError: Skipped 1 previous similar message [ 1319.769920] Lustre: server umount lustre-OST0000 complete [ 1325.542838] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1325.561085] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1325.588206] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 1325.597171] Lustre: Skipped 1 previous similar message [ 1329.843695] Lustre: server umount lustre-MDT0000 complete [ 1333.219736] LustreError: 76265:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1333.229211] LustreError: 76265:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 4 previous similar messages [ 1333.519757] LustreError: 77049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788904624 with bad export cookie 10133850152573115443 [ 1333.527045] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1333.531460] LustreError: 77049:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1333.844808] Lustre: server umount lustre-MDT0001 complete [ 1342.573980] Lustre: DEBUG MARKER: == conf-sanity test 80: mgc import reconnect race ======== 17:57:10 (1788904630) [ 1349.752912] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1350.341955] LustreError: 80636:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1350.470869] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1355.308854] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1355.749752] LustreError: 80637:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1360.869830] LustreError: 80636:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1364.760441] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1369.578406] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1378.196174] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1383.934345] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1391.542751] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1391.912756] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 1391.920812] Lustre: Skipped 1 previous similar message [ 1401.589485] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1413.102325] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1418.373397] Lustre: 83309:0:(genops.c:1803:obd_export_evict_by_uuid()) MGS: evicting 79d58df4-f586-43ae-8bba-5abc2f87d5cc at adminstrative request [ 1419.745635] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1419.764707] Lustre: *** cfs_fail_loc=906, val=2147483648*** [ 1429.997082] Lustre: MGS: Client 79d58df4-f586-43ae-8bba-5abc2f87d5cc (at 0@lo) reconnecting [ 1430.026204] Lustre: Evicted from MGS (at 0@lo) after server handle changed from 0x8ca2ab05b7289fa5 to 0x8ca2ab05b728a245 [ 1430.047946] Lustre: MGC192.168.201.106@tcp: Connection restored to 0@lo (at 0@lo) [ 1455.291140] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1455.480785] Lustre: lustre-OST0001: new disk, initializing [ 1455.486680] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1455.497161] Lustre: 83703:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1460.867325] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1460.880566] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1461.038330] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1461.338647] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1469.050661] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1476.578109] LustreError: lustre-OST0001-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1476.578979] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1476.595746] LustreError: Skipped 1 previous similar message [ 1476.601680] Lustre: Skipped 1 previous similar message [ 1476.608675] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 1476.615836] Lustre: Skipped 3 previous similar messages [ 1486.304139] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 1486.363863] Lustre: server umount lustre-OST0001 complete [ 1486.826712] LustreError: 82459:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1486.838821] LustreError: 82459:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1491.427112] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1491.436642] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1491.452993] Lustre: Skipped 1 previous similar message [ 1491.458562] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1491.465976] Lustre: Skipped 2 previous similar messages [ 1495.164416] Lustre: server umount lustre-OST0000 complete [ 1503.946547] Lustre: server umount lustre-MDT0000 complete [ 1506.306717] LustreError: 80617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788904796 with bad export cookie 10133850152573116997 [ 1506.307734] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1506.316160] LustreError: 80617:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 1506.505961] Lustre: server umount lustre-MDT0001 complete [ 1511.367861] Lustre: DEBUG MARKER: == conf-sanity test 81: sparse OST indexing ============== 18:00:00 (1788904800) [ 1512.296650] Lustre: DEBUG MARKER: SKIP: conf-sanity test_81 needs >= 3 OSTs [ 1513.238397] Lustre: DEBUG MARKER: == conf-sanity test 82a: specify OSTs for file (succeed) or directory (succeed) ========================================================== 18:00:02 (1788904802) [ 1514.187693] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82a needs >= 3 OSTs [ 1515.362872] Lustre: DEBUG MARKER: == conf-sanity test 82b: specify OSTs for file with --pool and --ost-list options ========================================================== 18:00:04 (1788904804) [ 1516.250324] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82b needs >= 4 OSTs [ 1517.402700] Lustre: DEBUG MARKER: == conf-sanity test 82c: specify sparse OSTs for setstripe (should not crash) ========================================================== 18:00:06 (1788904806) [ 1518.393594] Lustre: DEBUG MARKER: SKIP: conf-sanity test_82c needs >= 4 OSTs [ 1519.269388] Lustre: DEBUG MARKER: == conf-sanity test 83: ENOSPACE on OST doesn't cause message VFS: Busy inodes after unmount ... ========================================================== 18:00:08 (1788904808) [ 1520.405080] Lustre: DEBUG MARKER: mount the OST /dev/mapper/ost1_flakey as a ldiskfs filesystem [ 1524.005543] loop: module loaded [ 1524.029313] loop0: detected capacity change from 0 to 4294967296 [ 1524.074248] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 1524.929873] Lustre: DEBUG MARKER: run llverfs in partial mode on the OST ldiskfs /mnt/lustre-ost1 [ 1526.674713] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing run_llverfs /mnt/lustre-ost1 -vpl no [ 1528.446369] Lustre: DEBUG MARKER: unmount the OST /dev/mapper/ost1_flakey [ 1543.266497] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1543.298362] LustreError: 87821:0:(obd_config.c:845:class_setup()) setup lustre-OST0000-osd failed (-28) [ 1543.305408] LustreError: 87821:0:(obd_mount.c:259:lustre_start_simple()) lustre-OST0000-osd setup error -28 [ 1543.310510] LustreError: 87821:0:(tgt_mount.c:2671:server_fill_super()) Unable to start osd on /dev/mapper/ost1_flakey: -28 [ 1543.314603] LustreError: 87821:0:(super25.c:178:lustre_fill_super()) llite: Unable to mount : rc = -28 [ 1553.616518] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 1559.523963] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1579.519764] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1579.654792] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1579.665836] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1579.704537] Lustre: lustre-MDT0000: new disk, initializing [ 1579.772976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1579.776994] Lustre: Skipped 1 previous similar message [ 1579.785621] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1582.155642] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1588.536504] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1588.623420] Lustre: 90694:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1588.802975] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1591.213360] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1597.347439] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1600.321316] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1604.541848] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1604.727967] Lustre: 92660:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1608.438135] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1613.852152] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 1613.913709] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1614.560728] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1617.385597] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1617.517431] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1620.002422] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1620.150130] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1624.035080] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1624.035675] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1624.039556] LustreError: Skipped 1 previous similar message [ 1624.045328] Lustre: Skipped 3 previous similar messages [ 1624.048768] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1624.055517] Lustre: Skipped 5 previous similar messages [ 1627.427411] Lustre: server umount lustre-OST0000 complete [ 1635.830571] Lustre: server umount lustre-MDT0000 complete [ 1637.803359] LustreError: 90688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788904928 with bad export cookie 10133850152573117613 [ 1637.808276] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1637.810454] LustreError: 90688:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1638.020473] Lustre: server umount lustre-MDT0001 complete [ 1642.423761] Lustre: DEBUG MARKER: == conf-sanity test 84: check recovery_hard_time ========= 18:02:12 (1788904932) [ 1645.804310] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1646.054729] LustreError: 95070:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1646.064771] LustreError: 95070:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1648.297589] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1652.854077] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1655.353137] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1659.107677] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1661.652518] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1664.960356] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1668.195724] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1671.927643] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1673.805954] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1673.896686] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1676.053432] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1678.262522] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 2 sec [ 1681.834791] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1681.929154] Lustre: lustre-OST0001: new disk, initializing [ 1681.930750] Lustre: Skipped 2 previous similar messages [ 1681.932548] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 1681.934373] Lustre: Skipped 3 previous similar messages [ 1681.936833] Lustre: 98452:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1684.633260] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1689.630230] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 1689.633774] Lustre: Skipped 2 previous similar messages [ 1689.636438] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 1689.709426] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 1689.796641] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1691.742206] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid 50 [ 1691.833672] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1693.778618] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 1693.870461] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1705.024895] Lustre: DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 [ 1711.812809] Lustre: Failing over lustre-MDT0000 [ 1712.242629] Lustre: server umount lustre-MDT0000 complete [ 1714.656889] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 1714.660321] LustreError: Skipped 1 previous similar message [ 1714.662395] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1714.666927] Lustre: Skipped 2 previous similar messages [ 1714.668687] LustreError: 95070:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1714.674463] LustreError: 95070:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 1727.449289] LDISKFS-fs (dm-0): 4 truncates cleaned up [ 1727.451105] LDISKFS-fs (dm-0): recovery complete [ 1727.455456] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1727.494617] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1727.581976] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-60 [ 1727.584902] Lustre: Skipped 6 previous similar messages [ 1727.605543] Lustre: lustre-MDT0000: in recovery but waiting for the first client to connect [ 1728.965980] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1731.326136] Lustre: lustre-MDT0000: Will be in recovery for at least 1:00, or until 3 clients reconnect [ 1731.326963] Lustre: 95066:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1733.089561] Lustre: 95071:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 0 [ 1733.094043] Lustre: 95071:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 1 previous similar message [ 1733.098995] Lustre: lustre-MDT0000-lwp-OST0001: Connection restored to 0@lo (at 0@lo) [ 1733.115902] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1733.432100] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1733.754327] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1733.758070] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1734.072121] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1734.074529] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 1 previous similar message [ 1734.394509] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1734.398641] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 8 previous similar messages [ 1735.025966] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1735.028903] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1735.344083] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1735.346223] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 3 previous similar messages [ 1736.601038] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1736.604853] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 6 previous similar messages [ 1737.233077] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1737.235724] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1737.552097] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1737.554596] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 6 previous similar messages [ 1740.682557] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1740.686103] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 12 previous similar messages [ 1741.313775] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1741.317983] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1741.632078] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1741.634288] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 12 previous similar messages [ 1748.818141] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1748.821969] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 25 previous similar messages [ 1749.450317] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1749.453863] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1749.768128] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1749.771354] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 25 previous similar messages [ 1765.066393] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) lustre-MDT0000: extended recovery timer reached hard limit: 60, extend: 1 [ 1765.071249] Lustre: 101273:0:(ldlm_lib.c:2084:extend_recovery_timer()) Skipped 51 previous similar messages [ 1765.698444] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 sleeping for 300ms [ 1765.701648] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 51 previous similar messages [ 1766.016103] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) cfs_fail_timeout id 709 awake [ 1766.019552] LustreError: 101273:0:(ldlm_lib.c:2697:replay_request_or_update()) Skipped 51 previous similar messages [ 1791.296084] Lustre: 101273:0:(ldlm_lib.c:2393:target_recovery_overseer()) lustre-MDT0000 recovery is aborted by hard timeout [ 1791.299065] Lustre: 101273:0:(ldlm_lib.c:2403:target_recovery_overseer()) recovery is aborted, evict exports in recovery [ 1791.302038] Lustre: 101273:0:(genops.c:1630:class_disconnect_stale_exports()) lustre-MDT0000: disconnect stale client 0698f60d-fb2a-4c7a-ad26-12f9c0cdcc85@192.168.201.6@tcp [ 1791.305182] Lustre: lustre-MDT0000: disconnecting 1 stale clients [ 1791.306939] Lustre: 101273:0:(ldlm_lib.c:1913:abort_req_replay_queue()) @@@ aborted: req@00000000ce718528 x1875802087035904/t0(8589934783) o101->0698f60d-fb2a-4c7a-ad26-12f9c0cdcc85@192.168.201.6@tcp:542/0 lens 592/0 e 0 to 0 dl 1788905092 ref 1 fl Complete:/604/ffffffff rc 0/-1 job:'' uid:0 gid:0 projid:0 [ 1791.315023] Lustre: lustre-MDT0000-osd: cancel update llog [0x200000400:0x1:0x0] [ 1791.318157] Lustre: lustre-MDT0000: Denying connection for new client 0698f60d-fb2a-4c7a-ad26-12f9c0cdcc85 (at 192.168.201.6@tcp), waiting for 3 known clients (2 recovered, 0 in progress, and 1 evicted) to recover in 0:00 [ 1791.321302] Lustre: lustre-MDT0001-osp-MDT0000: cancel update llog [0x240000401:0x1:0x0] [ 1791.338107] Lustre: lustre-MDT0000-osp-MDT0001: Connection restored to 0@lo (at 0@lo) [ 1791.340703] Lustre: Skipped 2 previous similar messages [ 1791.342534] Lustre: lustre-MDT0000: Recovery over after 1:00, of 3 clients 2 recovered and 1 was evicted. [ 1791.357820] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:81 to 0x280000401:577) [ 1791.357830] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:48 to 0x2c0000401:545) [ 1797.089339] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 1797.091695] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1797.095513] Lustre: Skipped 3 previous similar messages [ 1797.097299] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1797.098930] Lustre: Skipped 3 previous similar messages [ 1802.939822] Lustre: server umount lustre-OST0000 complete [ 1804.769084] LustreError: 96892:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1804.774754] LustreError: 96892:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 19 previous similar messages [ 1810.319323] Lustre: server umount lustre-OST0001 complete [ 1817.694573] Lustre: server umount lustre-MDT0000 complete [ 1819.009141] LustreError: 95051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905109 with bad export cookie 10133850152573168811 [ 1819.011835] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1819.012405] LustreError: 95051:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 1819.124202] Lustre: server umount lustre-MDT0001 complete [ 1821.804404] Lustre: DEBUG MARKER: == conf-sanity test 85: osd_ost init: fail ea_fid_set ==== 18:05:11 (1788905111) [ 1824.456582] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1824.459172] Lustre: *** cfs_fail_loc=197, val=0*** [ 1839.904574] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 1845.024337] LustreError: 103245:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.106@tcp: failed processing log, type 4: rc = -110 [ 1850.150384] Lustre: *** cfs_fail_loc=197, val=0*** [ 1850.151772] Lustre: Skipped 65 previous similar messages [ 1872.505813] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1874.779165] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1875.446218] Lustre: server umount lustre-OST0000 complete [ 1878.120823] Lustre: DEBUG MARKER: == conf-sanity test 86: Replacing mkfs.lustre -G option == 18:06:07 (1788905167) [ 1891.437819] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1891.583974] LustreError: 104862:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 1891.590545] LustreError: 104862:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 1892.922623] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1896.525916] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1898.444323] LustreError: 104896:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0001-osp-MDT0000: osp_attr_get update error [0x200000009:0x1:0x0]: rc = -5 [ 1898.448221] LustreError: 104896:0:(lod_sub_object.c:917:lod_sub_prep_llog()) lustre-MDT0000-mdtlov: can't get id from catalogs: rc = -5 [ 1898.452099] LustreError: 104896:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 7, retries 0, failed: rc = -5 [ 1898.563335] Lustre: server umount lustre-MDT0000 complete [ 1908.950695] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 1912.173458] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1923.506108] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1923.581995] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1923.592215] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1923.626448] Lustre: lustre-MDT0000: new disk, initializing [ 1923.651381] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 1924.983097] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1929.048152] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1929.087604] Lustre: 109063:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 1929.186614] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 1930.485517] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1934.152213] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 1935.882299] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 1938.059128] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 1938.161206] Lustre: 111028:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 1940.150890] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 1940.320541] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 1944.060289] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 1945.656060] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 1945.743667] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 1947.325525] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 1947.413158] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 1950.691516] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 1950.691558] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 1950.694599] Lustre: Skipped 5 previous similar messages [ 1950.697331] LustreError: Skipped 1 previous similar message [ 1950.698600] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 1950.701807] Lustre: Skipped 10 previous similar messages [ 1954.196947] Lustre: server umount lustre-OST0000 complete [ 1961.559265] Lustre: server umount lustre-MDT0000 complete [ 1962.788111] LustreError: 109054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905253 with bad export cookie 10133850152573179535 [ 1962.791450] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 1962.792555] LustreError: 109054:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 1962.901619] Lustre: server umount lustre-MDT0001 complete [ 1965.596917] Lustre: DEBUG MARKER: == conf-sanity test 87: check if MDT inode can hold EAs with N stripes properly ========================================================== 18:07:35 (1788905255) [ 1968.452501] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 1969.440302] Key type lgssc unregistered [ 1969.560297] LNet: 113512:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 1969.562558] LNetError: 113512:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 1969.573318] LNet: Removed LNI 192.168.201.106@tcp [ 1969.873880] Key type .llcrypt unregistered [ 1969.874882] Key type ._llcrypt unregistered [ 1975.646118] Key type ._llcrypt registered [ 1975.647489] Key type .llcrypt registered [ 1975.685402] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 1981.824774] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 1982.242736] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 1982.250531] alg: No test for adler32 (adler32-zlib) [ 1983.099555] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 1983.183339] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 1984.760172] Key type lgssc registered [ 1985.138235] Lustre: Echo OBD driver; http://www.lustre.org/ [ 1998.379160] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 1998.385970] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 1999.471162] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 1999.482179] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 1999.520339] Lustre: lustre-MDT0000: new disk, initializing [ 1999.542610] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 1999.549263] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2000.924289] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2005.235033] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2005.330427] Lustre: lustre-OST0000: new disk, initializing [ 2005.333269] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2005.336489] Lustre: Skipped 1 previous similar message [ 2005.338900] Lustre: 117660:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2005.364570] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2006.078695] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 2006.081159] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 2006.089893] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 2007.266958] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2011.373340] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2011.433039] Lustre: lustre-OST0001: new disk, initializing [ 2011.434607] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 2011.437314] Lustre: 118523:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2011.462669] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 2012.306050] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:1:ost [ 2012.310377] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:1:ost] [ 2012.320711] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x280000400 [ 2013.557732] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2017.761034] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2017.761761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2017.764649] Lustre: Skipped 1 previous similar message [ 2017.768489] Lustre: Skipped 1 previous similar message [ 2022.881247] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2022.884501] Lustre: Skipped 1 previous similar message [ 2023.607185] Lustre: server umount lustre-MDT0000 complete [ 2025.765800] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [ 2027.323918] LustreError: 116846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905317 with bad export cookie 9802411445113590995 [ 2027.325738] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2027.327535] LustreError: 116846:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2027.359894] Lustre: server umount lustre-OST0000 complete [ 2028.690069] Lustre: server umount lustre-OST0001 complete [ 2031.521035] Lustre: DEBUG MARKER: == conf-sanity test 88a: tunefs.lustre persists a --mountfsoptions-only change ========================================================== 18:08:41 (1788905321) [ 2038.801384] Lustre: DEBUG MARKER: == conf-sanity test 88b: check the default mount options can be overridden ========================================================== 18:08:48 (1788905328) [ 2041.778210] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2049.117789] Lustre: DEBUG MARKER: == conf-sanity test 89: check tunefs --param and --erase-param{s} options ========================================================== 18:08:58 (1788905338) [ 2061.611088] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 2064.727019] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2076.081759] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2076.160699] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2076.170515] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2076.203741] Lustre: lustre-MDT0000: new disk, initializing [ 2076.224209] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2076.230714] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2077.526886] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2081.575195] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2081.619708] Lustre: 126257:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2081.643434] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2081.645954] Lustre: Skipped 1 previous similar message [ 2081.680879] Lustre: lustre-MDT0001: new disk, initializing [ 2081.712720] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2081.725252] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2081.728908] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 2083.028676] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2086.685369] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2088.468406] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL failover.node=192.0.2.254@tcp0,mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2090.589050] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2090.686558] Lustre: lustre-OST0000: new disk, initializing [ 2090.688820] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 2090.692107] Lustre: 128222:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 2090.719265] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2092.584677] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2094.011835] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 2094.015929] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 2094.046848] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 2096.301626] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) failover.node=192.0.2.254@tcp0,osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2097.592798] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 2097.660317] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 2098.863071] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 2098.926526] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 2104.292255] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2104.292499] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2104.294043] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2104.294581] LustreError: Skipped 1 previous similar message [ 2104.297541] Lustre: Skipped 1 previous similar message [ 2105.613228] Lustre: server umount lustre-OST0000 complete [ 2107.360885] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2107.363026] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2107.367373] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2107.369040] Lustre: Skipped 1 previous similar message [ 2109.409979] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2114.529037] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2114.531396] Lustre: Skipped 3 previous similar messages [ 2121.184148] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2121.237471] Lustre: server umount lustre-MDT0000 complete [ 2122.447098] LustreError: 126249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905413 with bad export cookie 9802411445113592206 [ 2122.450239] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2122.451344] LustreError: 126249:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2122.559304] Lustre: server umount lustre-MDT0001 complete [ 2125.193734] Lustre: DEBUG MARKER: == conf-sanity test 90a: check max_mod_rpcs_in_flight is enforced ========================================================== 18:10:15 (1788905415) [ 2127.178967] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2127.333378] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2128.552853] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2131.142807] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2132.459167] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2134.736758] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2136.490323] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2138.672702] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2140.635736] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2142.995642] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2155.746691] Lustre: *** cfs_fail_loc=159, val=0*** [ 2172.174369] Lustre: lustre-MDT0000: Client 2abe56b3-e6a9-45f4-abfd-5bc048423b08 (at 192.168.201.6@tcp) reconnecting [ 2172.660364] Lustre: *** cfs_fail_loc=159, val=0*** [ 2172.662288] Lustre: Skipped 5 previous similar messages [ 2189.067476] Lustre: lustre-MDT0000: Client 2abe56b3-e6a9-45f4-abfd-5bc048423b08 (at 192.168.201.6@tcp) reconnecting [ 2192.352753] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2192.354905] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2192.358798] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2192.361036] Lustre: Skipped 2 previous similar messages [ 2196.361700] Lustre: server umount lustre-OST0000 complete [ 2197.984799] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2197.986962] LustreError: Skipped 1 previous similar message [ 2203.668687] Lustre: server umount lustre-MDT0000 complete [ 2204.899226] LustreError: 130613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905495 with bad export cookie 9802411445113593074 [ 2204.902470] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2204.903050] LustreError: 130613:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2205.019482] Lustre: server umount lustre-MDT0001 complete [ 2208.467598] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2209.461506] Key type lgssc unregistered [ 2209.576280] LNet: 134425:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2209.578919] LNetError: 134425:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2209.588259] LNet: Removed LNI 192.168.201.106@tcp [ 2209.871537] Key type .llcrypt unregistered [ 2209.872625] Key type ._llcrypt unregistered [ 2218.394536] Key type ._llcrypt registered [ 2218.395964] Key type .llcrypt registered [ 2218.436347] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2218.829751] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2218.838265] alg: No test for adler32 (adler32-zlib) [ 2219.681791] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2219.764518] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2221.344143] Key type lgssc registered [ 2221.695310] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2224.322531] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2224.328809] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2225.475082] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2226.793805] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2229.487765] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2229.597101] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2230.811806] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2233.075910] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2234.816648] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2236.956734] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2237.066126] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2238.955058] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2241.297611] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2250.406567] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:42 to 0x280000401:65) [ 2256.436706] Lustre: *** cfs_fail_loc=159, val=0*** [ 2272.008711] Lustre: lustre-MDT0000: Client 7241a027-6bae-4bcb-b2e5-8b55a8f508bb (at 192.168.201.6@tcp) reconnecting [ 2272.774229] Lustre: *** cfs_fail_loc=159, val=0*** [ 2289.427315] Lustre: lustre-MDT0001: Client 7241a027-6bae-4bcb-b2e5-8b55a8f508bb (at 192.168.201.6@tcp) reconnecting [ 2290.259220] Lustre: *** cfs_fail_loc=159, val=0*** [ 2290.261732] Lustre: Skipped 3 previous similar messages [ 2305.803437] Lustre: lustre-MDT0001: Client 7241a027-6bae-4bcb-b2e5-8b55a8f508bb (at 192.168.201.6@tcp) reconnecting [ 2308.075179] Lustre: 138872:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2309.532417] Lustre: *** cfs_fail_loc=159, val=0*** [ 2309.534134] Lustre: Skipped 5 previous similar messages [ 2324.776121] Lustre: lustre-MDT0000: Client 1513fc83-cda3-4d55-86fd-208c92b8aeae (at 192.168.201.6@tcp) reconnecting [ 2325.701368] Lustre: *** cfs_fail_loc=159, val=0*** [ 2325.706053] Lustre: Skipped 15 previous similar messages [ 2342.185584] Lustre: lustre-MDT0000: Client 1513fc83-cda3-4d55-86fd-208c92b8aeae (at 192.168.201.6@tcp) reconnecting [ 2343.149456] Lustre: 139159:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2348.005061] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2348.007582] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2348.012022] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2348.015741] Lustre: Skipped 1 previous similar message [ 2348.243447] Lustre: server umount lustre-OST0000 complete [ 2352.609383] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2352.614116] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2352.622340] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2352.625510] Lustre: Skipped 1 previous similar message [ 2356.585773] Lustre: server umount lustre-MDT0000 complete [ 2358.240843] LustreError: 135834:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2358.251814] LustreError: 135834:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2358.442163] LustreError: 135796:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905649 with bad export cookie 2053698458588129216 [ 2358.446900] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2358.448030] LustreError: 135796:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 2358.628447] Lustre: server umount lustre-MDT0001 complete [ 2363.242688] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2364.520436] Key type lgssc unregistered [ 2364.681354] LNet: 140207:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2364.684808] LNetError: 140207:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2364.694355] LNet: Removed LNI 192.168.201.106@tcp [ 2365.101783] Key type .llcrypt unregistered [ 2365.103304] Key type ._llcrypt unregistered [ 2376.414658] Key type ._llcrypt registered [ 2376.416190] Key type .llcrypt registered [ 2376.470638] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2376.906806] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2376.940088] alg: No test for adler32 (adler32-zlib) [ 2377.835906] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2377.942472] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2379.536213] Key type lgssc registered [ 2379.997169] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2383.071795] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2383.079605] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2384.263937] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2385.861674] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2389.186548] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2389.338597] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2390.963553] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2393.906283] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2396.322231] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2399.295323] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2399.457412] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2402.027050] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2405.361083] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2408.685398] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:129) [ 2413.801530] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:97) [ 2417.982832] Lustre: 144211:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2418.713731] Lustre: 144259:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2419.246866] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 in log params [ 2420.664858] Lustre: 144409:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre/mdc.lustre-MDT0000-mdc-*.max_rpcs_in_flight=8 (mode = 0) failed: rc = -17 [ 2421.011464] Lustre: Modifying parameter lustre.mdc.lustre-MDT0000-mdc-*.max_mod_rpcs_in_flight=7 in log params [ 2421.014126] Lustre: Skipped 1 previous similar message [ 2421.386537] Lustre: 144507:0:(mdt_handler.c:64:mdt_max_mod_rpcs_per_client_set()) max_mod_rpcs_per_client is deprecated, set mdt.*.max_mod_rpcs_in_flight parameter instead [ 2424.291802] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2424.292638] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2424.293488] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2424.299468] LustreError: Skipped 1 previous similar message [ 2424.303859] Lustre: Skipped 1 previous similar message [ 2428.779281] Lustre: server umount lustre-OST0000 complete [ 2430.433684] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2430.436906] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2430.441138] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2430.444242] Lustre: Skipped 1 previous similar message [ 2434.529150] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2434.529945] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2436.581497] Lustre: server umount lustre-MDT0000 complete [ 2438.104414] LustreError: 141590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905728 with bad export cookie 10254019331325339377 [ 2438.107165] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2438.109275] LustreError: 141590:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2438.245273] Lustre: server umount lustre-MDT0001 complete [ 2442.593143] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2443.806113] Key type lgssc unregistered [ 2443.954471] LNet: 145563:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2443.958406] LNetError: 145563:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2443.966464] LNet: Removed LNI 192.168.201.106@tcp [ 2444.337835] Key type .llcrypt unregistered [ 2444.339628] Key type ._llcrypt unregistered [ 2455.093430] Key type ._llcrypt registered [ 2455.094721] Key type .llcrypt registered [ 2455.135372] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2455.606189] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2455.655799] alg: No test for adler32 (adler32-zlib) [ 2456.534841] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2456.636761] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2458.224160] Key type lgssc registered [ 2458.670956] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2461.783667] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2461.791685] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2462.975169] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2464.602482] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2468.053736] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2468.217742] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2469.915798] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2472.836845] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2475.160590] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2478.055347] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2478.199481] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2480.408698] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2483.328630] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2487.401071] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:85 to 0x280000401:161) [ 2492.519486] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:129) [ 2496.908301] Lustre: *** cfs_fail_loc=159, val=0*** [ 2513.164443] Lustre: lustre-MDT0000: Client df80f290-0da6-44ac-9beb-82e2a5808798 (at 192.168.201.6@tcp) reconnecting [ 2518.500591] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 2518.501349] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2518.502196] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2518.511756] Lustre: Skipped 1 previous similar message [ 2520.799811] Lustre: server umount lustre-OST0000 complete [ 2523.617483] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2523.622553] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2523.624807] Lustre: Skipped 2 previous similar messages [ 2528.478687] Lustre: server umount lustre-MDT0000 complete [ 2528.737280] LustreError: 147683:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2528.742772] LustreError: 147683:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2529.936269] LustreError: 149884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905820 with bad export cookie 16878253653443691389 [ 2529.938770] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2529.944507] LustreError: 149884:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 2530.103119] Lustre: server umount lustre-MDT0001 complete [ 2534.302335] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2535.465280] Key type lgssc unregistered [ 2535.594474] LNet: 150634:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2535.598284] LNetError: 150634:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2535.605408] LNet: Removed LNI 192.168.201.106@tcp [ 2535.991721] Key type .llcrypt unregistered [ 2535.993519] Key type ._llcrypt unregistered [ 2547.191331] Key type ._llcrypt registered [ 2547.192475] Key type .llcrypt registered [ 2547.239352] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2547.620996] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2547.647643] alg: No test for adler32 (adler32-zlib) [ 2548.526908] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2548.634208] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2550.224135] Key type lgssc registered [ 2550.683628] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2554.062628] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2554.068798] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2555.256824] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2556.933248] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2560.776857] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2560.943322] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2562.619044] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2566.095454] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2568.490833] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2571.362227] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2571.535305] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2573.974550] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2577.418827] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2579.789711] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:170 to 0x280000401:193) [ 2585.900269] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:161) [ 2608.152539] Lustre: lustre-OST0000: evicting 67caa9b5-bf4a-44bd-a9fa-866804106ab5 (at 192.168.201.6@tcp) by administrative request [ 2608.159373] Lustre: Skipped 1 previous similar message [ 2642.404470] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2642.413461] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2642.449255] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2647.529337] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2647.544607] Lustre: Skipped 1 previous similar message [ 2650.335248] Lustre: server umount lustre-OST0000 complete [ 2657.764126] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2657.767046] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2657.798389] Lustre: Skipped 2 previous similar messages [ 2657.847097] Lustre: Skipped 1 previous similar message [ 2662.892372] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2662.895966] Lustre: Skipped 1 previous similar message [ 2663.526424] Lustre: server umount lustre-MDT0000 complete [ 2668.004245] LustreError: 154757:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2668.057506] LustreError: 154757:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 2670.245194] LustreError: 152008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788905960 with bad export cookie 7245749116945464716 [ 2670.259262] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2670.277087] LustreError: 152008:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 2671.090362] Lustre: server umount lustre-MDT0001 complete [ 2689.805337] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2693.086740] Key type lgssc unregistered [ 2693.436843] LNet: 156139:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2693.441625] LNetError: 156139:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2693.458535] LNet: Removed LNI 192.168.201.106@tcp [ 2694.359374] Key type .llcrypt unregistered [ 2694.364497] Key type ._llcrypt unregistered [ 2756.113696] Key type ._llcrypt registered [ 2756.116664] Key type .llcrypt registered [ 2756.196703] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2757.234102] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2757.277565] alg: No test for adler32 (adler32-zlib) [ 2758.263574] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2758.451844] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2760.104157] Key type lgssc registered [ 2761.194721] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2768.145743] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2768.162212] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2769.588520] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2773.725631] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2780.130022] LustreError: 158606:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2780.158965] LustreError: 158606:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 2782.373263] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2782.846965] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2787.654992] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2794.717317] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 2799.574295] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 2806.482496] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 2806.948397] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 2807.989835] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:72 to 0x280000400:193) [ 2812.905789] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2815.174186] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:195 to 0x280000401:225) [ 2819.547873] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 2828.769918] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 2828.780300] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2828.804302] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2830.827221] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2830.838368] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 2830.863586] Lustre: Skipped 1 previous similar message [ 2834.533481] Lustre: server umount lustre-OST0000 complete [ 2839.009453] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 2839.016893] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2839.030618] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2841.058962] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2841.063763] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 2846.177718] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 2846.185755] Lustre: Skipped 2 previous similar messages [ 2852.322045] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 2852.522718] Lustre: server umount lustre-MDT0000 complete [ 2855.401201] LustreError: 158601:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 2855.418776] LustreError: 158601:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 2855.441814] LustreError: 158586:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788906146 with bad export cookie 5031489740867457555 [ 2855.450089] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 2855.668039] Lustre: server umount lustre-MDT0001 complete [ 2867.490214] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 2870.068509] Key type lgssc unregistered [ 2870.328065] LNet: 162149:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 2870.343842] LNetError: 162149:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 2870.362952] LNet: Removed LNI 192.168.201.106@tcp [ 2871.473670] Key type .llcrypt unregistered [ 2871.475535] Key type ._llcrypt unregistered [ 2896.006927] Key type ._llcrypt registered [ 2896.011468] Key type .llcrypt registered [ 2896.119382] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2897.208992] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 2897.252086] alg: No test for adler32 (adler32-zlib) [ 2898.379456] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 2898.574089] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 2900.216251] Key type lgssc registered [ 2901.180367] Lustre: Echo OBD driver; http://www.lustre.org/ [ 2914.324515] Lustre: DEBUG MARKER: == conf-sanity test 100: check lshowmount lists MGS, MDT, OST and 0@lo ========================================================== 18:23:23 (1788906203) [ 2931.760536] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 2941.670404] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 2979.715130] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 2979.762469] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2981.104226] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 2981.152841] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 2981.332683] Lustre: lustre-MDT0000: new disk, initializing [ 2981.417792] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 2981.445601] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 2985.936237] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 2996.415451] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 2996.537665] Lustre: 166412:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 2996.587341] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 2996.590839] Lustre: Skipped 1 previous similar message [ 2996.693541] Lustre: lustre-MDT0001: new disk, initializing [ 2996.835178] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 2996.896181] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 2996.916369] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 3002.032267] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 3012.418959] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3017.979704] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3024.959355] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3025.195870] Lustre: lustre-OST0000: new disk, initializing [ 3025.199687] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 3025.208701] Lustre: 168381:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 3025.321841] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3031.066120] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 3035.226944] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 3035.248658] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 3035.373524] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 3040.277508] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3050.993938] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3050.999672] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 3051.032333] LustreError: Skipped 1 previous similar message [ 3051.035890] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 3051.042798] Lustre: Skipped 1 previous similar message [ 3054.609704] Lustre: server umount lustre-OST0000 complete [ 3058.148436] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 3058.163192] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3058.189427] Lustre: Skipped 1 previous similar message [ 3058.202207] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3061.221137] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 3061.225349] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3064.172371] Lustre: server umount lustre-MDT0000 complete [ 3067.456792] LustreError: 166404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788906358 with bad export cookie 16544559550103009359 [ 3067.465250] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 3067.476356] LustreError: 166404:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 3067.949426] Lustre: server umount lustre-MDT0001 complete [ 3078.812313] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 3081.608941] Key type lgssc unregistered [ 3081.883866] LNet: 170282:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 3081.892814] LNetError: 170282:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 3081.910821] LNet: Removed LNI 192.168.201.106@tcp [ 3082.776538] Key type .llcrypt unregistered [ 3082.779844] Key type ._llcrypt unregistered [ 3106.822589] Key type ._llcrypt registered [ 3106.824890] Key type .llcrypt registered [ 3107.029728] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 3108.206712] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 3108.404743] alg: No test for adler32 (adler32-zlib) [ 3109.651388] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 3109.964818] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 3111.642570] Key type lgssc registered [ 3112.976951] Lustre: Echo OBD driver; http://www.lustre.org/ [ 3120.233347] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 3120.249431] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3121.566543] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 3126.270205] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 3131.877964] LustreError: 171696:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3131.891813] LustreError: 171696:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 3136.998826] LustreError: 171697:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 3137.591739] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 3138.691290] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 3147.566719] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 3158.662432] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 3166.358621] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 3174.436586] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 3174.712390] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 3181.736625] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 3190.480275] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 3192.117443] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:3 to 0x280000401:33) [ 3202.286896] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3202.335686] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3202.367243] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3202.390555] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3202.411819] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3203.740473] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3203.755841] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3203.763776] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3203.770704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3203.776896] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3204.957900] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3204.974731] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3204.984050] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3204.988978] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3205.007154] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3206.134971] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.388040] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3207.396977] Lustre: Skipped 1 previous similar message [ 3207.411963] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3207.420832] Lustre: Skipped 1 previous similar message [ 3207.424140] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3207.429818] Lustre: Skipped 1 previous similar message [ 3207.433943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3207.443674] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3207.457789] Lustre: Skipped 1 previous similar message [ 3208.677612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3209.659129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3210.678772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.493422] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3211.498703] Lustre: Skipped 3 previous similar messages [ 3211.514741] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3211.526173] Lustre: Skipped 3 previous similar messages [ 3211.532790] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3211.537865] Lustre: Skipped 3 previous similar messages [ 3211.541190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3211.553151] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3211.559655] Lustre: Skipped 3 previous similar messages [ 3212.519853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3213.606548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3214.763616] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3215.977355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.004840] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3217.988956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3218.857178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.927652] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3219.933979] Lustre: Skipped 7 previous similar messages [ 3219.951836] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3219.959211] Lustre: Skipped 7 previous similar messages [ 3219.962871] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3219.967946] Lustre: Skipped 7 previous similar messages [ 3219.971665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3219.995996] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3220.010873] Lustre: Skipped 7 previous similar messages [ 3220.738466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3221.777703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3222.696618] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3223.705378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3224.580717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3225.755354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3226.671052] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3227.591233] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3228.518771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3229.466956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3230.492384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3231.290370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3232.266702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3233.092179] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.003703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3234.888898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3235.928766] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3235.935308] Lustre: Skipped 16 previous similar messages [ 3235.955533] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3235.965640] Lustre: Skipped 16 previous similar messages [ 3235.969072] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3235.971483] Lustre: Skipped 16 previous similar messages [ 3235.973677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.844991] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3236.855976] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3236.859315] Lustre: Skipped 17 previous similar messages [ 3237.935293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3239.001037] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3240.067792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3241.093582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3242.198856] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3243.077448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3244.172685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3245.417685] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3246.358891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3247.385941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3248.333952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3249.228808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.023524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3250.939417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3251.978306] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3252.893846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3253.678314] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3254.812465] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3255.668751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3256.691758] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3257.652103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3259.092449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3260.427871] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3261.469292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3262.532836] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3263.546950] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3264.512781] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3265.447753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3266.548657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3267.666844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3268.609838] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3268.614550] Lustre: Skipped 31 previous similar messages [ 3268.645544] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3268.665754] Lustre: Skipped 31 previous similar messages [ 3268.675592] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3268.687534] Lustre: Skipped 31 previous similar messages [ 3268.700449] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.775682] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3269.792727] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3269.808444] Lustre: Skipped 31 previous similar messages [ 3270.769043] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3271.532385] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3272.476518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3273.535336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3274.450773] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3275.734290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.087699] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3277.947582] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3278.929634] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3279.765081] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3280.733216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3281.647846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3282.719713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3283.821132] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3284.937784] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3286.200511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3287.814416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3289.242401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3290.487987] LustreError: 171716:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3290.500386] LustreError: 171716:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3290.507257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3291.856934] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3292.974339] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3293.784844] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3294.769561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3295.661838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3296.715396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3297.779337] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3298.823569] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3299.569466] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3300.450413] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3301.481194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3302.393417] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3303.610655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3304.902203] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3306.261115] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3307.363436] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3308.410970] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3309.384166] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3310.107694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.064782] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3311.944416] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3312.888962] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3313.886351] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3314.913942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3315.761913] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3316.620647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3317.466498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3318.481723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3319.278375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3320.230707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.069095] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3321.876264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3322.675021] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3323.586434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3324.546492] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3325.382573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3326.206882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3327.114456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.054587] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3328.936346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3329.799607] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3330.695955] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3331.539819] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3332.628423] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3332.637703] Lustre: Skipped 63 previous similar messages [ 3332.652892] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3332.659837] Lustre: Skipped 63 previous similar messages [ 3332.664650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3333.560282] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3333.564199] Lustre: Skipped 64 previous similar messages [ 3333.566539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.442562] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3334.447841] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3334.450706] Lustre: Skipped 64 previous similar messages [ 3335.517656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3336.373644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3337.499593] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3338.624071] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3339.486072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3340.423196] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3341.286586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3342.208612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.152450] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3343.973364] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3344.838380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3345.787848] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3346.627129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3347.614426] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3348.520883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3349.587706] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3350.527572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3351.525561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3352.476133] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3353.387845] LustreError: 171716:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3353.398405] LustreError: 171716:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3353.420488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3354.329909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3355.286329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3356.403162] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3357.292681] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3358.293352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3359.158116] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3360.137464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3361.071250] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3362.171734] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3363.119845] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3364.235742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3365.219639] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3366.182355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3367.038896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3368.226625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3369.446595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3370.876592] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3372.115345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3373.303690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3374.307772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3375.384600] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3376.510896] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3377.524909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3378.557620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3379.408296] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3380.272768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3381.242049] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3382.345863] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3383.341093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3384.358702] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.224301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3385.961943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3386.908208] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3387.783163] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3388.797867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3389.602300] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3390.450898] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3391.310216] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.142073] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3392.900688] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3393.834320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3394.895788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3395.718987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3396.603583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3397.607409] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3398.534615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3399.475669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3400.348194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3401.257346] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3402.252379] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3403.143402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3404.203650] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3405.089264] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.016545] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3406.815657] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3407.735889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3408.657958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3409.675437] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3410.749307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3411.736171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3412.620907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3413.592282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3414.545195] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3415.540412] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3416.466298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3417.389031] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3418.421943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3419.317455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3420.261622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3421.141210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3422.143524] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3423.091333] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3424.156277] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3425.254889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3426.389289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3427.355788] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3428.610191] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3429.775368] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3430.615209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3431.708679] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3432.614768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3433.494986] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3434.531585] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3435.409464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3436.373344] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3437.325112] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3438.267684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3439.288627] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3440.243612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3441.280397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3442.121486] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3443.113521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.163761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3444.917904] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3445.967554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3446.846355] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3447.941477] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3448.799872] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3449.802283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3450.529239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3451.375439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3452.300252] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3453.169695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3454.026263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3455.116658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3456.002286] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3457.091924] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.054410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3458.997604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3459.994076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3460.961615] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3460.967219] Lustre: Skipped 132 previous similar messages [ 3460.982613] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3460.991135] Lustre: Skipped 132 previous similar messages [ 3460.994752] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3461.962453] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3461.972607] Lustre: Skipped 132 previous similar messages [ 3461.980861] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3462.993399] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3463.006873] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3463.011177] Lustre: Skipped 132 previous similar messages [ 3464.101425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3465.221332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3466.122967] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.049738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3467.881812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3468.937192] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3469.898771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3470.747265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3471.729556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3472.729656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3473.719952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3474.630275] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3475.456882] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3476.363584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3477.266186] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.218675] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3478.990994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3479.883251] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3480.697105] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3481.513953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3482.470247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3483.179485] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.048701] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3484.998748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3486.312457] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3487.510748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3488.345414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3489.203606] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3490.051153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3491.188130] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3492.152290] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3493.092659] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3494.102508] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3495.241401] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3496.162843] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3497.227341] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3498.281673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3499.659410] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.016536] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3501.967088] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3502.903866] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3503.673769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3504.687525] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3505.655220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3506.478488] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3507.445415] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3508.256222] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.054748] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3509.888498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3510.660745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3511.564830] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3512.392285] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.142576] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3513.847388] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3514.680620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3515.561080] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3516.474398] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3517.303880] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3518.316868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3519.381596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3520.432354] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3521.382102] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3522.537714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3523.657900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3524.641318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3525.700802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3526.550072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3527.575153] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3528.375020] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3529.250567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.025515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3530.874187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3531.755336] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3532.641833] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3533.431123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3534.366800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3535.226114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.075097] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3536.883443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3537.741876] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3538.524034] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3539.511766] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3540.277114] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3541.156736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3542.076212] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3543.131491] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3544.061755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.087293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3545.938690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3546.833961] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3547.721899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3548.648886] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3549.639745] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3550.731451] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3551.636636] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3552.566604] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3553.438100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3554.384620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3555.307979] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.105383] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3556.879974] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3557.811666] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3558.622499] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3559.456789] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3560.242456] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3561.141498] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3561.985074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3562.842144] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3563.649641] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3564.507757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3565.455603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3566.596257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3567.597253] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3568.710738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3569.736994] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3570.618059] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3571.614518] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3572.628382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3573.664503] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3574.608588] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3575.710180] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3576.727647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3577.647421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3578.384441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3579.472443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3580.395857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3581.418093] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3582.271510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.251307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3583.999846] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3584.929149] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3585.964230] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3587.113220] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3588.070070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3589.161772] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.150637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3590.926857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3591.612205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3592.472944] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3593.348586] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3594.224661] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3595.216074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3596.229143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3597.051663] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3597.847434] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3598.752316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3599.560085] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3600.469556] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3601.327156] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3602.197905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3603.128200] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3604.312348] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3605.073311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.086521] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3606.927318] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3607.794794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3608.826968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3609.778283] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3610.535165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3611.483673] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3612.327860] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3613.250635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3614.066287] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.022881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3615.854484] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3616.755763] LustreError: 171716:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3616.763564] LustreError: 171716:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3616.770278] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3617.671316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3618.703452] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3619.742279] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3620.504617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3621.324952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.107537] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3622.760481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3623.547380] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3624.345134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3625.248774] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.003827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3626.934490] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3627.970473] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3628.873178] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3629.824695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3630.595580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3631.473032] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3632.138762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3633.111583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3634.072742] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.015891] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3635.949516] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3636.753072] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3637.579219] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3638.463338] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3639.238862] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.060182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3640.737610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3641.554543] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3642.360931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.162464] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3643.853674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3644.689311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3645.421798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.212589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3646.927378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3647.722119] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3648.423168] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.153448] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3649.941850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.767885] LustreError: 171716:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -5 [ 3650.776932] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3650.779581] LustreError: 171716:0:(osp_precreate.c:1359:osp_precreate_thread()) lustre-OST0000-osc-MDT0000: cannot precreate objects: rc = -5 [ 3651.501878] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3652.424741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3653.301124] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.158225] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3654.858181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3655.677440] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3656.500320] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3657.266732] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.071953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3658.886838] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3659.478391] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.095458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3660.898369] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3661.525609] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.138689] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3662.809202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3663.509722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.281694] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3664.946751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3665.573122] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.203319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3666.856841] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3667.571249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3668.272989] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.048396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3669.716919] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3670.423165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3671.280528] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.070832] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3672.699199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3673.499638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3674.349611] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.113809] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3675.867262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3676.674301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3677.398316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.113504] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3678.832343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3679.520643] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.269613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3680.965857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3681.715719] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3682.409960] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.200631] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3683.779247] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3684.552316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.126977] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3685.901769] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3686.591568] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3687.283430] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3688.056382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3689.100345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.109120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3690.946199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3691.752763] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3692.888352] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3693.758753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3694.801056] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3695.684578] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3696.566797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3697.379265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.110684] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3698.938931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3699.692717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3700.361045] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.147188] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3701.795889] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3702.591749] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3703.432079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.157397] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3704.829542] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3705.395714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.192964] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3706.979992] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3707.858033] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3708.758217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3709.588700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3710.351952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.072738] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3711.808621] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3712.532644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.173644] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3713.813965] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3714.627027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3715.458483] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3716.378271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3717.164421] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 3717.167317] Lustre: Skipped 295 previous similar messages [ 3717.188548] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 3717.193900] Lustre: Skipped 295 previous similar messages [ 3717.197726] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.059561] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 3718.063589] Lustre: Skipped 295 previous similar messages [ 3718.066386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3718.842425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.847612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3719.858311] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 3719.861591] Lustre: Skipped 296 previous similar messages [ 3720.620074] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3721.509850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3722.378858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.212649] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3723.896942] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3724.617842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3725.354013] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.064359] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3726.809370] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3727.623589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3728.327626] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.033272] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3729.706691] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3730.367239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3731.115651] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3731.889678] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3732.625990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3733.380660] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.181786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3734.809943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3735.646612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3736.541539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3737.398282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.211424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3738.937658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3739.699079] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.387539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3740.961712] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3741.739918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3742.405906] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3743.324553] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3744.494968] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3745.635360] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3746.633075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3747.610532] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3748.438100] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3749.284075] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.008280] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3750.894455] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3751.628768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3752.442425] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.131471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3753.848946] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3754.566329] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.211470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3755.929446] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3756.715150] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3757.397523] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.123808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3758.765210] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.336181] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3759.985744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3760.627764] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3761.329907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.042865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3762.750343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3763.471746] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3764.317874] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.022176] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3765.752638] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3766.357170] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.079740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3767.714392] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3768.384552] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.159853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3769.996343] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3770.669083] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3771.356165] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3772.143332] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.012273] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3773.732282] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3774.606146] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3775.444476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.277548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3776.974519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3777.732117] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3778.463943] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.185400] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3779.766229] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3780.553384] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3781.352231] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.134750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3782.862683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3783.575837] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3784.429850] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3785.272744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3786.100602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.019759] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3787.866747] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3788.746648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3789.804583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3790.664129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3791.439899] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.220803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3792.977753] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3793.855433] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3794.861713] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3795.671213] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3796.597292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3797.597547] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3798.678427] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3799.365622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.122036] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3800.787236] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3801.551674] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.213700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3802.951441] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3803.566389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3804.404564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3805.197771] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.119076] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3806.960207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3807.940558] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3808.943741] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.028507] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3810.869187] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3811.693311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3812.469640] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3813.293554] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.153381] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3814.911257] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3815.704164] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3816.411386] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.116082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3817.817475] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3818.597907] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3819.490160] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3820.283276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3821.117762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.042683] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3822.775301] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3823.464479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3824.269482] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.037821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3825.831202] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3826.560307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.244304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3827.948707] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3828.735510] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3829.456584] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.233602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3830.868615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3831.640956] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3832.334311] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.046373] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3833.743129] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3834.549945] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3835.426135] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3836.661597] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3837.541767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3838.662620] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3839.575799] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3840.403647] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.123335] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3841.939792] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3842.695603] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3843.428646] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.165509] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3844.993443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3845.889502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3846.681824] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3847.586740] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.257561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3848.982281] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3849.902972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3850.838656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3851.833708] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3852.753431] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3853.551926] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3854.403842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3855.213548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3856.130768] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3857.034560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.019476] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3858.821596] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3859.718304] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3860.441402] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3861.322842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.003642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3862.800723] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3863.494217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.378614] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3864.986755] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3865.746131] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3866.661729] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3867.701865] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3868.501883] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3869.499265] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3870.384779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.159894] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3871.864984] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3872.689658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3873.541293] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3874.539183] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3875.523794] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3876.603577] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3877.341969] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3878.310716] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3879.163724] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3880.311665] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3881.286209] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3882.254406] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3883.279065] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3884.146895] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3885.208655] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.036722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3886.922546] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3887.829704] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3888.854321] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3889.555583] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3890.681652] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3891.843711] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3892.800829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3893.809929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3894.708717] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3895.660767] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3896.478776] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3897.370445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3898.171375] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.098929] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3899.940497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3900.893718] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3901.735622] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3902.632953] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3903.451561] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3904.375262] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3905.153362] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3906.046751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.050345] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3907.994669] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3908.723757] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3909.652307] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3910.435511] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3911.359070] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3912.130811] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.010458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3913.839648] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3914.862424] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3915.782632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3916.904920] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3917.870040] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3918.888439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3919.740990] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3920.752909] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3921.957954] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3922.937439] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3924.160814] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3925.008270] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3926.318423] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3927.490690] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3928.381808] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.304571] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3929.997292] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.013215] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3931.964274] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.051927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3933.888241] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3934.907827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3935.936098] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3936.907802] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3938.006111] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3939.057589] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3940.243727] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3941.150342] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.059103] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3942.717612] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3943.487560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3944.148677] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.096572] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3945.850414] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3946.711199] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3947.628605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3948.646443] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3949.417530] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.232389] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3950.865695] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3951.608817] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.240573] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3952.983930] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3953.746500] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3954.447803] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3955.172207] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.016217] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3956.725908] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3957.639560] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3958.321952] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.067605] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3959.944948] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3960.752829] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3961.381637] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.262108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3962.960316] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3963.750881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3964.579770] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3965.449610] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3966.165762] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.008171] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3967.808047] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3968.871987] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3969.669190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3970.618182] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3971.387319] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3972.339931] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.013868] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3973.806489] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3974.618239] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3975.487590] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3976.240918] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3977.129092] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3978.182857] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3979.338595] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3980.442123] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3981.314728] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3982.370299] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3983.456142] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3984.609470] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3985.658642] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3986.547194] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3987.795260] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3988.735800] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3990.729869] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3992.223580] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3993.739327] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3994.884798] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3996.690853] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 3998.596276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4000.026539] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4001.466298] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4004.782602] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4006.313921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4007.631635] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4009.305382] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4011.687842] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4013.048458] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4014.034120] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4015.038821] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4016.326474] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4017.456567] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4018.304680] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4019.290550] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4020.131140] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4021.136693] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4022.030958] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4023.053263] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4024.058143] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.075289] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4025.950900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4026.846625] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4027.880720] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4028.779787] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4029.768108] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4030.820881] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4031.753538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4032.678790] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4033.557613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4034.513760] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4035.429591] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4036.432867] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4037.508471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4038.572453] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4039.480276] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4040.490765] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4041.398214] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4042.447407] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4043.468497] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4044.569797] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4045.298658] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4046.191501] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.051025] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4047.898995] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4048.821134] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4049.636827] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4050.466598] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4051.442110] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4052.222812] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4053.185927] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.118744] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4054.951459] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4055.840892] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4056.706261] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4057.565761] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4058.434445] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4059.461519] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4060.234779] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.024670] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4061.807471] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4062.722613] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4063.401921] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4064.397972] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.053481] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4065.749077] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4066.387905] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4067.186751] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4068.096828] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.037082] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4069.841900] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4070.709564] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4071.751700] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4073.071941] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4074.261714] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4075.152667] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.130777] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4076.989432] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4077.952515] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4079.026027] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.125632] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4080.960858] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.050570] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4082.846541] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4083.737378] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4084.568736] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4085.563479] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4086.553238] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4087.514538] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4088.567266] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4089.544211] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4090.630421] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4091.595703] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4092.536242] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4093.317190] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4094.375271] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4095.246668] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.205050] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4096.953438] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4098.004205] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4099.072249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4100.308656] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4101.203526] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4102.290877] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4103.202502] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4104.212617] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.110615] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4105.968722] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4106.739249] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4107.635786] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4108.443104] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4109.489548] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4110.456750] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4425.844854] LustreError: 173520:0:(ofd_dev.c:1780:ofd_create_hdl()) lustre-OST0000: unable to precreate: rc = -28 [ 4425.855709] LustreError: 171716:0:(osp_precreate.c:655:osp_precreate_send()) lustre-OST0000-osc-MDT0000: can't precreate: rc = -28 [ 4781.539321] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4781.544244] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4781.549967] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4781.566743] Lustre: Skipped 442 previous similar messages [ 4786.659573] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4786.665586] Lustre: Skipped 1 previous similar message [ 4786.997546] Lustre: server umount lustre-OST0000 complete [ 4791.779855] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4791.786271] Lustre: Skipped 1 previous similar message [ 4796.042687] Lustre: server umount lustre-MDT0000 complete [ 4796.906168] LustreError: 172424:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 4796.925676] LustreError: 172424:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 4798.896622] LustreError: 171676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908089 with bad export cookie 9779245645204831070 [ 4798.900918] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4798.904989] LustreError: 171676:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4799.191278] Lustre: server umount lustre-MDT0001 complete [ 4806.480708] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 4808.411763] Key type lgssc unregistered [ 4808.641978] LNet: 226896:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4808.648751] LNetError: 226896:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4808.662278] LNet: Removed LNI 192.168.201.106@tcp [ 4809.336535] Key type .llcrypt unregistered [ 4809.339890] Key type ._llcrypt unregistered [ 4828.013329] Key type ._llcrypt registered [ 4828.015568] Key type .llcrypt registered [ 4828.122949] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 4829.054331] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 4829.088036] alg: No test for adler32 (adler32-zlib) [ 4830.120772] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 4830.371250] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 4832.041081] Key type lgssc registered [ 4832.871860] Lustre: Echo OBD driver; http://www.lustre.org/ [ 4838.469465] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 4838.489745] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4839.821273] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 4842.952923] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 4849.097538] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 4849.452607] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 4852.410082] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 4858.046309] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 4862.411390] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 4868.399878] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4868.663103] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4873.825584] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 4879.868149] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4886.076293] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49665 to 0x280000401:49696) [ 4889.576491] Lustre: setting import lustre-OST0000_UUID INACTIVE by administrator request [ 4890.782770] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4890.793390] Lustre: lustre-OST0000: Client lustre-MDT0000-mdtlov_UUID (at 0@lo) reconnecting [ 4890.799396] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4890.813798] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4890.817684] LustreError: 228310:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 sleeping for 20000ms [ 4895.206474] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4895.212841] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4895.222274] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4896.226241] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4896.228170] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4896.247075] Lustre: Skipped 1 previous similar message [ 4898.457516] Lustre: server umount lustre-OST0000 complete [ 4910.824213] LustreError: 228310:0:(osp_precreate.c:1268:osp_precreate_thread()) cfs_fail_timeout id 2107 awake [ 4930.269343] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 4930.586347] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 4934.748201] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 4936.745029] LustreError: lustre-OST0000-osc-MDT0000: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4936.754212] Lustre: lustre-OST0000-osc-MDT0000: Connection restored to 0@lo (at 0@lo) [ 4939.809190] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 4940.862415] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid in FULL state after 0 sec [ 4944.419811] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cb182a4b000.ost_server_uuid 50 [ 4945.625346] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cb182a4b000.ost_server_uuid in FULL state after 0 sec [ 4946.980612] LustreError: lustre-OST0000-osc-MDT0001: This client was evicted by lustre-OST0000; in progress operations using this service will fail. [ 4946.991939] Lustre: lustre-OST0000-osc-MDT0001: Connection restored to 0@lo (at 0@lo) [ 4952.033800] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 4952.043498] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4952.049727] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 4952.062564] Lustre: Skipped 1 previous similar message [ 4952.072773] Lustre: Skipped 2 previous similar messages [ 4954.755911] Lustre: server umount lustre-OST0000 complete [ 4962.276549] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 4962.276612] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 4962.279933] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 4962.295860] Lustre: Skipped 1 previous similar message [ 4963.650817] Lustre: server umount lustre-MDT0000 complete [ 4966.292872] LustreError: 228271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908256 with bad export cookie 5621212194727003367 [ 4966.295091] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 4966.305233] LustreError: 228271:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 4966.504898] Lustre: server umount lustre-MDT0001 complete [ 4974.060787] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 4975.966942] Key type lgssc unregistered [ 4976.191862] LNet: 233495:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 4976.200382] LNetError: 233495:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 4977.258503] LNet: Removed LNI 192.168.201.106@tcp [ 4977.950276] Key type .llcrypt unregistered [ 4977.952446] Key type ._llcrypt unregistered [ 5000.884145] Key type ._llcrypt registered [ 5000.887323] Key type .llcrypt registered [ 5000.971331] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5001.908254] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5001.979904] alg: No test for adler32 (adler32-zlib) [ 5003.028034] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 5003.230652] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 5004.896383] Key type lgssc registered [ 5005.989409] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5013.213076] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5013.229486] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5014.629055] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5017.838169] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5024.683632] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5024.743962] LustreError: 235201:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5024.762410] LustreError: 235201:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5024.995256] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5028.172674] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5030.369346] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 5035.532078] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5035.781309] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5039.863655] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5040.902873] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49698 to 0x280000401:49728) [ 5047.063959] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5047.303323] Lustre: lustre-OST0001: new disk, initializing [ 5047.307870] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5047.317197] Lustre: 237508:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5047.411847] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5047.722919] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5047.728168] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5047.817135] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5052.455897] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5061.393219] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5067.644428] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 5076.449320] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5076.453594] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5076.464205] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5078.582250] Lustre: server umount lustre-MDT0000 complete [ 5080.546038] LustreError: 236001:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5080.552220] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5080.568438] LustreError: 236001:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5080.598762] Lustre: Skipped 1 previous similar message [ 5081.477783] LustreError: 238450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908372 with bad export cookie 5940257194663948887 [ 5081.479031] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5081.484032] LustreError: 238450:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 5081.859591] Lustre: server umount lustre-MDT0001 complete [ 5094.960203] Lustre: server umount lustre-OST0000 complete [ 5108.221036] Lustre: server umount lustre-OST0001 complete [ 5124.176168] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5131.697440] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5131.954022] LustreError: 240614:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5131.973255] LustreError: 240614:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5132.019894] Lustre: mylustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5135.238899] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5137.385336] LustreError: 240615:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5137.398896] LustreError: 240615:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5141.349699] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5141.480972] Lustre: mylustre-MDT0001: Not available for connect from 0@lo (not set up) [ 5141.685966] Lustre: mylustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5144.779520] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5146.877696] Lustre: 241755:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5151.775851] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5153.061970] LustreError: 242108:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5156.819848] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5157.183550] Lustre: mylustre-OST0000: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49730 to 0x280000401:49760) [ 5162.469360] LustreError: 242110:0:(ldlm_lib.c:1192:target_handle_connect()) mylustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5162.491879] LustreError: 242110:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 5164.015390] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5164.192954] Lustre: mylustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 5164.195760] Lustre: Skipped 1 previous similar message [ 5168.528857] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5169.654461] Lustre: mylustre-OST0001: new connection from mylustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 5174.330184] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5176.770621] Lustre: 243637:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5185.001097] Lustre: mylustre-MDT0000-lwp-OST0000: Connection to mylustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5185.004714] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5185.034600] Lustre: Skipped 4 previous similar messages [ 5185.045908] Lustre: Skipped 3 previous similar messages [ 5190.116708] Lustre: mylustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5190.128231] Lustre: Skipped 3 previous similar messages [ 5190.390510] Lustre: server umount mylustre-MDT0000 complete [ 5193.119939] LustreError: 242935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908483 with bad export cookie 5940257194663951673 [ 5193.122304] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5193.131401] LustreError: 242935:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5193.349483] Lustre: server umount mylustre-MDT0001 complete [ 5206.388564] Lustre: server umount mylustre-OST0000 complete [ 5219.378616] Lustre: server umount mylustre-OST0001 complete [ 5232.112639] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5237.972826] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5238.210450] LustreError: 245942:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5238.257782] Lustre: tfs-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5240.828697] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5245.802777] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5248.733726] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5250.448571] Lustre: 247086:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5254.522410] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5258.306745] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5263.137561] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5264.319996] Lustre: tfs-OST0001: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 5264.341397] Lustre: tfs-OST0000: new connection from tfs-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49792) [ 5266.977879] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5272.918602] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5276.486760] Lustre: 248955:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5286.900183] LustreError: tfs-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5286.948373] Lustre: tfs-MDT0000-osp-MDT0001: Connection to tfs-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5286.961747] Lustre: tfs-MDT0000: Not available for connect from 0@lo (stopping) [ 5291.651751] Lustre: server umount tfs-MDT0000 complete [ 5293.030710] LustreError: 246676:0:(ldlm_lib.c:1192:target_handle_connect()) tfs-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5293.087831] LustreError: 246676:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 6 previous similar messages [ 5297.259553] LustreError: 245918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908587 with bad export cookie 5940257194663953570 [ 5297.266266] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5297.284037] LustreError: 245918:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5297.784179] Lustre: server umount tfs-MDT0001 complete [ 5313.337617] Lustre: server umount tfs-OST0000 complete [ 5313.376097] Lustre: 234468:0:(client.c:2490:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1788908588/real 1788908588] req@ffff888504a36680 x1875806302428416/t0(0) o400->tfs-MDT0001-lwp-OST0001@0@lo:12/10 lens 224/224 e 0 to 1 dl 1788908604 ref 1 fl Rpc:XNQr/200/ffffffff rc 0/-1 job:'kworker.0' uid:0 gid:0 projid:4294967295 [ 5313.388859] Lustre: tfs-MDT0001-lwp-OST0001: Connection to tfs-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5313.396727] Lustre: Skipped 3 previous similar messages [ 5316.602184] Lustre: server umount tfs-OST0001 complete [ 5343.876752] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5354.224686] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5354.936818] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5354.948440] Lustre: Skipped 3 previous similar messages [ 5359.886692] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5360.097959] LustreError: 251204:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5360.117349] LustreError: 251204:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 5367.331732] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5373.084465] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5375.984732] Lustre: 252345:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 5381.805719] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5387.033614] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5395.127603] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5399.478140] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 5399.491826] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:49762 to 0x280000401:49824) [ 5401.038650] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5407.863182] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 5410.850645] Lustre: 254216:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 5416.390585] Lustre: DEBUG MARKER: == conf-sanity test 104a: Make sure user defined options are reflected in mount ========================================================== 19:05:05 (1788908705) [ 5434.337357] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5434.346854] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5434.368519] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5434.377383] Lustre: Skipped 4 previous similar messages [ 5445.088113] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5445.580561] Lustre: server umount lustre-MDT0000 complete [ 5449.449963] LustreError: 251179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908740 with bad export cookie 5940257194663955236 [ 5449.460747] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5449.470234] LustreError: 251179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 5450.731485] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5450.744073] Lustre: Skipped 4 previous similar messages [ 5450.744854] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 5450.755373] Lustre: Skipped 9 previous similar messages [ 5456.164363] Lustre: server umount lustre-MDT0001 complete [ 5465.574772] Lustre: server umount lustre-OST0000 complete [ 5475.439538] Lustre: server umount lustre-OST0001 complete [ 5481.018926] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 5490.109609] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5520.824807] LDISKFS-fs (dm-0): Mount option "noacl" will be removed by 3.5 [ 5520.824807] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5520.824807] [ 5520.884155] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5521.111661] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5521.127142] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5521.185291] Lustre: lustre-MDT0000: new disk, initializing [ 5521.237939] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5521.246267] Lustre: Skipped 3 previous similar messages [ 5521.263402] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5524.708492] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5533.023344] LDISKFS-fs (dm-1): Mount option "noacl" will be removed by 3.5 [ 5533.023344] Contact linux-ldiskfs@vger.kernel.org if you think we should keep it. [ 5533.023344] [ 5533.073387] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: acl,user_xattr,errors=remount-ro,noacl,no_mbcache,nodelalloc [ 5533.147882] Lustre: 257402:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5533.191392] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 5533.195836] Lustre: Skipped 1 previous similar message [ 5533.267621] Lustre: lustre-MDT0001: new disk, initializing [ 5533.422933] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 5533.431873] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5537.307811] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5546.118559] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5546.342272] Lustre: lustre-OST0000: new disk, initializing [ 5546.347340] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5546.351569] Lustre: 258988:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5547.604236] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5547.608591] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5547.678064] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5551.464211] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5560.198964] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5560.339656] Lustre: lustre-OST0001: new disk, initializing [ 5560.342780] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 5560.347760] Lustre: 259857:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5562.253237] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 5562.260934] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 5562.351872] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 5564.556929] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5573.908246] Lustre: DEBUG MARKER: == conf-sanity test 104b: Mount uses last flock argument ========================================================== 19:07:42 (1788908862) [ 5587.695423] Lustre: DEBUG MARKER: == conf-sanity test 105: check file creation for ro and rw bind mnt pt ========================================================== 19:07:56 (1788908876) [ 5603.316810] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5603.327038] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5603.331306] Lustre: Skipped 1 previous similar message [ 5603.344247] Lustre: Skipped 3 previous similar messages [ 5616.096928] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 5616.178833] Lustre: server umount lustre-OST0000 complete [ 5618.668424] LustreError: 258971:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5618.678477] LustreError: 258971:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 11 previous similar messages [ 5620.705760] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5625.721187] Lustre: server umount lustre-MDT0000 complete [ 5629.054223] LustreError: 258197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788908919 with bad export cookie 5940257194663956790 [ 5629.056367] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5629.067435] LustreError: 258197:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 5635.422855] Lustre: server umount lustre-MDT0001 complete [ 5644.441632] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 5652.379083] Lustre: server umount lustre-OST0001 complete [ 5656.586440] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 5667.976149] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5696.841754] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5697.098244] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 5697.118098] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 5697.176614] Lustre: lustre-MDT0000: new disk, initializing [ 5697.233886] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 5700.557668] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5708.512180] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5708.603902] Lustre: 264387:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 5708.773849] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 5711.753919] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5718.373663] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5723.108503] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5729.926064] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5730.260461] Lustre: lustre-OST0000: new disk, initializing [ 5730.264552] Lustre: Skipped 1 previous similar message [ 5730.269460] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 5730.272286] Lustre: Skipped 2 previous similar messages [ 5730.276728] Lustre: 266359:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 5731.880369] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 5731.891447] Lustre: Skipped 1 previous similar message [ 5731.897308] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 5732.009307] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 5736.057799] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5743.494840] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5752.289795] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5752.295517] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5752.300799] Lustre: Skipped 4 previous similar messages [ 5752.304411] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5752.307044] Lustre: Skipped 9 previous similar messages [ 5755.278271] Lustre: server umount lustre-OST0000 complete [ 5764.364293] Lustre: server umount lustre-MDT0000 complete [ 5767.406276] LustreError: 264380:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909058 with bad export cookie 5940257194663958260 [ 5767.421526] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5767.771853] Lustre: server umount lustre-MDT0001 complete [ 5774.558097] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 5776.437270] Key type lgssc unregistered [ 5776.647527] LNet: 268116:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5776.650656] LNetError: 268116:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5776.673694] LNet: Removed LNI 192.168.201.106@tcp [ 5777.278322] Key type .llcrypt unregistered [ 5777.281734] Key type ._llcrypt unregistered [ 5794.841486] Key type ._llcrypt registered [ 5794.843378] Key type .llcrypt registered [ 5794.935942] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5795.600232] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5795.652117] alg: No test for adler32 (adler32-zlib) [ 5796.604798] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 5796.757843] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 5798.408169] Key type lgssc registered [ 5799.123890] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5803.957404] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5803.967371] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5805.250248] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 5808.311491] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5814.760594] LustreError: 269558:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 5814.775874] LustreError: 269558:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 5815.015400] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 5815.265255] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 5818.124640] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5823.510328] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 5827.748354] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 5832.693305] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5833.010135] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5836.958358] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5842.464281] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5843.307881] Lustre: Setting parameter lustre-OST0000.ost.unknown_param=50 in log lustre-OST0000 [ 5845.479879] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5845.485756] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 5845.490264] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 5847.316907] Lustre: server umount lustre-OST0000 complete [ 5851.105555] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 5851.117059] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5851.129120] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5854.694844] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 5854.695389] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 5854.713616] Lustre: Skipped 1 previous similar message [ 5856.519569] Lustre: server umount lustre-MDT0000 complete [ 5859.149169] LustreError: 269540:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909149 with bad export cookie 11668150762245890385 [ 5859.158297] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 5859.385389] Lustre: server umount lustre-MDT0001 complete [ 5867.939977] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 5870.001652] Key type lgssc unregistered [ 5870.189966] LNet: 273136:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5870.197585] LNetError: 273136:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5870.209634] LNet: Removed LNI 192.168.201.106@tcp [ 5870.755682] Key type .llcrypt unregistered [ 5870.757540] Key type ._llcrypt unregistered [ 5883.030592] Key type ._llcrypt registered [ 5883.033597] Key type .llcrypt registered [ 5883.113300] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5883.910025] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5883.935020] alg: No test for adler32 (adler32-zlib) [ 5884.961968] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 5885.165227] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 5886.864093] Key type lgssc registered [ 5887.671810] Lustre: Echo OBD driver; http://www.lustre.org/ [ 5893.618214] Lustre: lustre-OST0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 5893.631538] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 5909.088526] LustreError: Failed to get MGS log lustre-sptlrpc and no local copy. [ 5914.208403] LustreError: 274310:0:(mgc_request.c:233:do_config_log_add()) MGC192.168.201.106@tcp: failed processing log, type 4: rc = -110 [ 5940.897106] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 5945.009598] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 5950.928725] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 5952.599400] Lustre: server umount lustre-OST0000 complete [ 5965.164465] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 5967.140453] Key type lgssc unregistered [ 5967.327817] LNet: 275939:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 5967.333729] LNetError: 275939:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 5968.360204] LNet: Removed LNI 192.168.201.106@tcp [ 5968.932778] Key type .llcrypt unregistered [ 5968.935330] Key type ._llcrypt unregistered [ 5995.553940] Key type ._llcrypt registered [ 5995.555269] Key type .llcrypt registered [ 5995.628151] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 5996.115457] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 5996.166244] alg: No test for adler32 (adler32-zlib) [ 5997.165513] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 5997.320586] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 5998.944263] Key type lgssc registered [ 5999.733198] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6008.469104] loop0: detected capacity change from 0 to 204800000 [ 6008.593678] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6008.599449] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6008.798399] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6009.633866] loop0: detected capacity change from 0 to 204800000 [ 6009.733043] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6009.744708] blk_update_request: operation not supported error, dev loop0, sector 208 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6009.967884] blk_update_request: operation not supported error, dev loop0, sector 160232 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6010.916730] loop0: detected capacity change from 0 to 204800000 [ 6011.054826] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6011.065301] blk_update_request: operation not supported error, dev loop0, sector 8224 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6011.151214] blk_update_request: operation not supported error, dev loop0, sector 58264 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6011.876199] loop0: detected capacity change from 0 to 204800000 [ 6012.045342] blk_update_request: operation not supported error, dev loop0, sector 399872 op 0x9:(WRITE_ZEROES) flags 0x400800 phys_seg 0 prio class 0 [ 6012.874227] loop0: detected capacity change from 0 to 204800000 [ 6012.910060] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6015.552918] loop0: detected capacity change from 0 to 204800000 [ 6015.573790] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6018.145117] loop0: detected capacity change from 0 to 204800000 [ 6018.174319] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6020.653351] loop0: detected capacity change from 0 to 204800000 [ 6020.669415] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: (null) [ 6023.079767] loop0: detected capacity change from 0 to 204800000 [ 6023.134175] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6023.155134] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6023.198318] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6023.203070] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index 'nodemap' with [0x200000003:0x3:0x0]: rc = 0 [ 6023.208936] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000-MDT0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6023.215338] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000003:0xc:0x0]: rc = 0 [ 6023.223661] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000-MDT0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6023.229410] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000003:0xb:0x0]: rc = 0 [ 6023.238229] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000003:0xa:0x0]: rc = 0 [ 6023.245405] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000-MDT0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6023.252889] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1020000' with [0x200000006:0x1020000:0x0]: rc = 0 [ 6023.259389] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2020000' with [0x200000006:0x2020000:0x0]: rc = 0 [ 6023.265831] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x20000' with [0x200000006:0x20000:0x0]: rc = 0 [ 6023.270744] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x2010000' with [0x200000006:0x2010000:0x0]: rc = 0 [ 6023.276299] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x1010000' with [0x200000006:0x1010000:0x0]: rc = 0 [ 6023.282991] Lustre: 279329:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0000: restore index '0x10000' with [0x200000006:0x10000:0x0]: rc = 0 [ 6027.259674] Lustre: server umount MGS complete [ 6027.841060] loop0: detected capacity change from 0 to 204800000 [ 6027.913894] LDISKFS-fs (loop0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6028.259322] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6028.292621] Lustre: lustre-MDT0000: Aborting client recovery [ 6028.863277] loop1: detected capacity change from 0 to 204800000 [ 6028.916462] LDISKFS-fs (loop1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6029.001438] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6029.010187] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000' with [0x200000003:0x4:0x0]: rc = 0 [ 6029.018597] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000' with [0x200000003:0x6:0x0]: rc = 0 [ 6029.027051] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000-MDT0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6029.044967] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x2010000-MDT0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6029.050966] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x1010000' with [0x200000003:0x5:0x0]: rc = 0 [ 6029.060867] Lustre: 279760:0:(osd_scrub.c:1854:osd_index_restore()) lustre-MDT0001: restore index '0x10000-MDT0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6029.078171] Lustre: Found index 1 for lustre-MDT0001, updating log [ 6029.201924] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6029.234231] Lustre: lustre-MDT0001: Aborting client recovery [ 6029.745854] loop2: detected capacity change from 0 to 204800000 [ 6029.810769] LDISKFS-fs (loop2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6029.900215] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6029.908827] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6029.914453] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x20000-OST0000' with [0x200000005:0x1:0x0]: rc = 0 [ 6029.925487] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6029.937076] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6029.947207] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x1020000-OST0000' with [0x200000005:0x2:0x0]: rc = 0 [ 6029.954870] Lustre: 279841:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0000: restore index '0x2020000-OST0000' with [0x200000005:0x3:0x0]: rc = 0 [ 6030.064812] Lustre: Found index 0 for lustre-OST0000, updating log [ 6030.094810] Lustre: 279859:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6030.144487] Lustre: lustre-OST0000: Aborting recovery [ 6030.737207] loop3: detected capacity change from 0 to 204800000 [ 6030.805315] LDISKFS-fs (loop3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6030.881969] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index 'fld' with [0x200000001:0x3:0x0]: rc = 0 [ 6030.888945] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000-OST0001' with [0x200000005:0x2:0x0]: rc = 0 [ 6030.902186] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000-OST0001' with [0x200000005:0x3:0x0]: rc = 0 [ 6030.915036] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000-OST0001' with [0x200000005:0x1:0x0]: rc = 0 [ 6030.923889] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x1020000' with [0x200000003:0x6:0x0]: rc = 0 [ 6030.932089] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x20000' with [0x200000003:0x5:0x0]: rc = 0 [ 6030.937542] Lustre: 279924:0:(osd_scrub.c:1854:osd_index_restore()) lustre-OST0001: restore index '0x2020000' with [0x200000003:0x7:0x0]: rc = 0 [ 6030.975418] Lustre: 279929:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6031.027297] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 6031.029235] Lustre: Skipped 1 previous similar message [ 6031.042401] Lustre: lustre-OST0001: Aborting recovery [ 6036.468439] LustreError: 279704:0:(lod_dev.c:511:lod_sub_recovery_thread()) lustre-MDT0001-osp-MDT0000: get update log duration 9, retries 0, failed: rc = -108 [ 6037.481880] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6037.489197] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x0:6 to 0x0:33) [ 6037.516080] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000bd0 [ 6037.544907] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000bd0 [ 6038.501726] Lustre: lustre-OST0001: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x2c0000400:5 to 0x2c0000400:33) [ 6038.501851] Lustre: lustre-OST0000: new connection from lustre-MDT0001-mdtlov (cleaning up unused objects from 0x280000400:5 to 0x280000400:33) [ 6038.530407] LustreError: 279670:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0000: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0000 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0000/identity_upcall, time 280us: rc = -2 [ 6039.031164] LustreError: 279667:0:(mdt_identity.c:115:mdt_identity_do_upcall()) lustre-MDT0001: error invoking upcall /root/Work/Lustre/R5/lustre-release/lustre/tests/../utils/l_getidentity lustre-MDT0001 0: rc -2; check /proc/fs/lustre/mdt/lustre-MDT0001/identity_upcall, time 707us: rc = -2 [ 6039.041654] LustreError: 279667:0:(mdt_identity.c:115:mdt_identity_do_upcall()) Skipped 36 previous similar messages [ 6053.857085] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6053.860177] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6053.866635] Lustre: Skipped 3 previous similar messages [ 6053.878769] Lustre: Skipped 3 previous similar messages [ 6056.870202] Lustre: server umount lustre-MDT0000 complete [ 6057.385204] LustreError: 279652:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909348 with bad export cookie 17215883283658538543 [ 6057.393363] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6058.976938] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6058.978847] LustreError: 280155:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6058.979022] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6058.989424] Lustre: Skipped 1 previous similar message [ 6059.016086] LustreError: 280155:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 6063.009602] LustreError: 279669:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6063.011807] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 6063.022687] Lustre: Skipped 2 previous similar messages [ 6063.739224] Lustre: server umount lustre-MDT0001 complete [ 6070.570401] Lustre: server umount lustre-OST0000 complete [ 6077.478904] Lustre: server umount lustre-OST0001 complete [ 6083.385651] Lustre: DEBUG MARKER: == conf-sanity test 109a: test lctl clear_conf fsname ==== 19:16:12 (1788909372) [ 6101.070609] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 6108.480970] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 6133.285921] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6133.448990] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6133.470581] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6133.551776] Lustre: lustre-MDT0000: new disk, initializing [ 6133.616820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6133.631436] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6136.594438] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6145.576214] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6145.655201] Lustre: 283993:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6145.704619] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6145.707212] Lustre: Skipped 1 previous similar message [ 6145.769444] Lustre: lustre-MDT0001: new disk, initializing [ 6145.871410] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6145.918080] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6145.928581] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6150.449685] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6157.557187] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6161.344910] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6165.986854] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6166.201870] Lustre: lustre-OST0000: new disk, initializing [ 6166.204727] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6166.209069] Lustre: 285960:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6166.263055] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6171.569831] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6172.199410] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6172.211034] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6172.312137] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6178.544451] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6182.457870] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6191.168716] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6200.282669] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6210.550752] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6243.810930] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6243.815614] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6243.820952] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6243.831308] Lustre: Skipped 1 previous similar message [ 6248.801801] Lustre: server umount lustre-OST0000 complete [ 6253.541786] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6253.548316] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6253.556267] Lustre: Skipped 1 previous similar message [ 6253.560677] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6264.289890] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6264.292756] Lustre: Skipped 5 previous similar messages [ 6266.336418] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 6266.477813] Lustre: server umount lustre-MDT0000 complete [ 6269.062393] LustreError: 283987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909559 with bad export cookie 17215883283658541252 [ 6269.065173] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6269.071887] LustreError: 283987:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 6 previous similar messages [ 6269.302665] Lustre: server umount lustre-MDT0001 complete [ 6274.129288] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6276.574262] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6279.822431] Lustre: server umount MGS complete [ 6287.379915] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6287.644713] LustreError: 289972:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6287.655664] LustreError: 289972:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6287.661415] LustreError: 289985:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6287.709049] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6290.388118] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6292.967030] LustreError: 289973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6296.169786] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6299.180532] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6303.681625] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6306.808146] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6310.811326] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6314.335241] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6318.327896] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6338.018360] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6338.024754] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6338.035238] Lustre: Skipped 1 previous similar message [ 6338.040689] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6343.097346] Lustre: server umount lustre-OST0000 complete [ 6347.745110] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6347.750284] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6347.762352] Lustre: Skipped 1 previous similar message [ 6351.605526] Lustre: server umount lustre-MDT0000 complete [ 6353.378215] LustreError: 289973:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6353.391273] LustreError: 289973:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6353.919272] LustreError: 289998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909644 with bad export cookie 17215883283658543198 [ 6353.924380] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6353.926865] LustreError: 289998:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6354.113655] Lustre: server umount lustre-MDT0001 complete [ 6360.652642] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 6362.247374] Key type lgssc unregistered [ 6362.468598] LNet: 293912:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6362.472869] LNetError: 293912:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6363.494811] LNet: Removed LNI 192.168.201.106@tcp [ 6364.036160] Key type .llcrypt unregistered [ 6364.038429] Key type ._llcrypt unregistered [ 6384.658804] Key type ._llcrypt registered [ 6384.660254] Key type .llcrypt registered [ 6384.725355] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 6393.457638] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 6394.027289] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6394.117259] alg: No test for adler32 (adler32-zlib) [ 6395.032957] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 6395.160358] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 6396.776256] Key type lgssc registered [ 6397.371277] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6416.185044] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6416.200796] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6417.365905] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6417.385790] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6417.430080] Lustre: lustre-MDT0000: new disk, initializing [ 6417.489114] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6417.505831] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6419.649582] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6426.115606] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6426.213275] Lustre: 297847:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 6426.268267] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 6426.272506] Lustre: Skipped 1 previous similar message [ 6426.339536] Lustre: lustre-MDT0001: new disk, initializing [ 6426.402324] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6426.422374] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 6426.430501] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 6428.928396] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6434.662695] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6437.884627] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6441.758615] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6441.939581] Lustre: lustre-OST0000: new disk, initializing [ 6441.942083] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6441.946819] Lustre: 299810:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6442.004710] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6445.422146] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6449.159760] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 6449.172224] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 6449.223921] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 6450.927398] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6456.866879] Lustre: Setting parameter lustre-MDT0000.mdd.atime_diff=62 in log lustre-MDT0000 [ 6460.804553] Lustre: Modifying parameter lustre-MDT0000.mdd.atime_diff=63 in log lustre-MDT0000 [ 6466.725752] Lustre: Setting parameter lustre-client.llite.max_read_ahead_mb=32 in log lustre-client [ 6473.669250] Lustre: Modifying parameter lustre-client.llite.max_read_ahead_mb=64 in log lustre-client [ 6515.682900] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6515.683920] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6515.685257] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6515.698106] Lustre: Skipped 1 previous similar message [ 6519.207881] Lustre: server umount lustre-OST0000 complete [ 6523.873936] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 6523.877644] LustreError: Skipped 1 previous similar message [ 6523.879307] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6523.884305] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6523.887247] Lustre: Skipped 1 previous similar message [ 6525.921617] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6525.924976] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6527.361421] Lustre: server umount lustre-MDT0000 complete [ 6529.398652] LustreError: 299809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909820 with bad export cookie 5185395672638216115 [ 6529.402398] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6529.404818] LustreError: 299809:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6529.572235] Lustre: server umount lustre-MDT0001 complete [ 6534.097596] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6535.878764] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6538.064084] Lustre: server umount MGS complete [ 6543.443027] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6543.726706] LustreError: 303591:0:(qmt_pool.c:1645:qmt_pool_add_rem()) lustre-QMT0000: can't remove lustre-OST0000_UUID pool 'pool1': rc = -22 [ 6543.758587] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6546.014728] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6548.961237] LustreError: 303579:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6548.969650] LustreError: 303579:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 6550.387278] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6550.552991] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 6552.691694] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6556.315898] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 6559.283829] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 6562.938453] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6563.122506] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6566.235102] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6569.873070] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 6596.072210] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 6596.072496] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6596.074750] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 6596.074755] Lustre: Skipped 1 previous similar message [ 6596.088809] Lustre: Skipped 1 previous similar message [ 6597.621850] Lustre: server umount lustre-OST0000 complete [ 6601.187142] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 6601.193301] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 6601.197118] Lustre: Skipped 2 previous similar messages [ 6605.659729] Lustre: server umount lustre-MDT0000 complete [ 6606.306631] LustreError: 303578:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 6606.317361] LustreError: 303578:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 6607.239931] LustreError: 303559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788909897 with bad export cookie 5185395672638217942 [ 6607.242198] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 6607.249699] LustreError: 303559:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 6607.381654] Lustre: server umount lustre-MDT0001 complete [ 6611.794129] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 6613.017832] Key type lgssc unregistered [ 6613.163662] LNet: 307521:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 6613.168186] LNetError: 307521:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 6613.177332] LNet: Removed LNI 192.168.201.106@tcp [ 6613.585705] Key type .llcrypt unregistered [ 6613.587901] Key type ._llcrypt unregistered [ 6623.394498] Key type ._llcrypt registered [ 6623.396119] Key type .llcrypt registered [ 6623.461951] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 6630.970859] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 6631.384388] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 6631.395206] alg: No test for adler32 (adler32-zlib) [ 6632.280699] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 6632.388032] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 6633.984175] Key type lgssc registered [ 6634.538048] Lustre: Echo OBD driver; http://www.lustre.org/ [ 6645.693274] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 6651.820072] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 6651.828828] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 6652.936707] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 6652.951382] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 6652.987279] Lustre: lustre-MDT0000: new disk, initializing [ 6653.022286] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 6653.035109] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 6654.997649] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 6660.147549] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 6660.268191] Lustre: lustre-OST0000: new disk, initializing [ 6660.270594] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 6660.272483] Lustre: Skipped 1 previous similar message [ 6660.274880] Lustre: 311546:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 6660.320870] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 6661.208276] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:0:ost [ 6661.213706] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:0:ost] [ 6661.231490] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x240000400 [ 6662.903983] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7029.483586] Lustre: server umount lustre-MDT0000 complete [ 7030.921569] LustreError: 310729:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910321 with bad export cookie 322364728412222233 [ 7030.926251] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7040.558161] Lustre: server umount lustre-OST0000 complete [ 7051.770923] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 7055.827487] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7071.351172] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7071.457100] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7071.470964] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7071.511478] Lustre: lustre-MDT0000: new disk, initializing [ 7071.545585] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7071.558486] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7073.402688] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7078.561587] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7078.628576] Lustre: 315800:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7078.659720] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7078.662183] Lustre: Skipped 1 previous similar message [ 7078.703278] Lustre: lustre-MDT0001: new disk, initializing [ 7078.763223] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7078.782749] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7078.787414] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7080.558432] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7085.582640] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7087.853329] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7090.512545] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7090.617558] Lustre: lustre-OST0000: new disk, initializing [ 7090.620257] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7090.623697] Lustre: 317767:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7090.655058] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7092.981862] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7097.771585] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7099.399125] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7099.402708] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7099.432843] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7099.544532] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7099.666881] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7101.532455] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7101.620178] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7104.484153] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7104.484972] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7104.493503] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7104.495558] LustreError: Skipped 1 previous similar message [ 7104.499149] Lustre: Skipped 1 previous similar message [ 7108.441929] Lustre: server umount lustre-OST0000 complete [ 7114.722252] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7114.724044] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7114.724761] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7114.727171] Lustre: Skipped 2 previous similar messages [ 7116.128173] Lustre: server umount lustre-MDT0000 complete [ 7117.532645] LustreError: 315791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910408 with bad export cookie 322364728415163010 [ 7117.533287] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7117.538973] LustreError: 315791:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7117.661216] Lustre: server umount lustre-MDT0001 complete [ 7119.123325] Lustre: DEBUG MARKER: SKIP: conf-sanity test_111 skipping SLOW test 111 [ 7119.863565] Lustre: DEBUG MARKER: == conf-sanity test 112a: mount OST with no_create option ========================================================== 19:33:29 (1788910409) [ 7122.510954] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7122.727525] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7124.349922] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7127.984282] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7128.033038] LustreError: 320077:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7128.039657] LustreError: 320077:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7129.916110] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7132.828873] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7135.144497] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7137.784532] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7140.238199] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7143.022425] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7145.783361] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7145.853137] Lustre: lustre-OST0001: new disk, initializing [ 7145.855439] Lustre: srv-lustre-OST0001: No data found on store. Initialize space. [ 7145.859300] Lustre: 322912:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7145.892244] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7145.895093] Lustre: Skipped 2 previous similar messages [ 7148.254811] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7148.679235] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x00000002c0000400-0x0000000300000400]:1:ost [ 7148.684177] Lustre: cli-lustre-OST0001-super: Allocated super-sequence [0x00000002c0000400-0x0000000300000400]:1:ost] [ 7148.705075] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7148.709770] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7149.734051] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7149.738284] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7150.759034] LustreError: 323341:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7150.763020] LustreError: 323341:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 1 previous similar message [ 7152.289654] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7152.373134] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7152.803088] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0001: cannot cleanup orphans: rc = -1 [ 7152.807685] LustreError: 323334:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 3 previous similar messages [ 7154.299912] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0000-osc-ffff9cb1a1769800.ost_server_uuid 50 [ 7154.855804] Lustre: DEBUG MARKER: osc.lustre-OST0000-osc-ffff9cb1a1769800.ost_server_uuid in FULL state after 0 sec [ 7156.769570] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state (FULL|IDLE) osc.lustre-OST0001-osc-ffff9cb1a1769800.ost_server_uuid 50 [ 7156.899298] LustreError: 323341:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) lustre-OST0001-osc-MDT0000: cannot cleanup orphans: rc = -1 [ 7156.903441] LustreError: 323341:0:(osp_precreate.c:974:osp_precreate_cleanup_orphans()) Skipped 7 previous similar messages [ 7157.300581] Lustre: DEBUG MARKER: osc.lustre-OST0001-osc-ffff9cb1a1769800.ost_server_uuid in FULL state after 0 sec [ 7157.931724] Lustre: lustre-OST0001-osc-MDT0000: update sequence from 0x100010000 to 0x2c0000401 [ 7170.535315] Lustre: lustre-OST0001-osc-MDT0001: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7170.536702] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7170.539099] Lustre: Skipped 1 previous similar message [ 7170.543534] Lustre: Skipped 2 previous similar messages [ 7175.652236] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7175.654324] Lustre: Skipped 2 previous similar messages [ 7180.772054] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7180.774158] Lustre: Skipped 2 previous similar messages [ 7183.328120] Lustre: lustre-OST0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7183.369101] Lustre: server umount lustre-OST0001 complete [ 7185.377258] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7185.381406] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7185.890810] LustreError: 321897:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7185.897297] LustreError: 321897:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7191.011325] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7191.011704] LustreError: 321896:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7191.014370] Lustre: Skipped 4 previous similar messages [ 7191.020474] LustreError: 321896:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7191.456191] Lustre: server umount lustre-OST0000 complete [ 7194.592811] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7194.595553] LustreError: Skipped 1 previous similar message [ 7194.597153] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7194.602098] Lustre: Skipped 1 previous similar message [ 7207.392190] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7207.459688] Lustre: server umount lustre-MDT0000 complete [ 7208.976212] LustreError: 320059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910499 with bad export cookie 322364728415163878 [ 7208.978983] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7208.980743] LustreError: 320059:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 3 previous similar messages [ 7209.123781] Lustre: server umount lustre-MDT0001 complete [ 7213.570644] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 7214.733556] Key type lgssc unregistered [ 7214.875450] LNet: 325590:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7214.878638] LNetError: 325590:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7214.886399] LNet: Removed LNI 192.168.201.106@tcp [ 7215.255607] Key type .llcrypt unregistered [ 7215.256962] Key type ._llcrypt unregistered [ 7225.770371] Key type ._llcrypt registered [ 7225.771413] Key type .llcrypt registered [ 7225.812434] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7226.277843] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7226.286121] alg: No test for adler32 (adler32-zlib) [ 7227.160495] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 7227.263056] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 7228.856211] Key type lgssc registered [ 7229.245566] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7232.194950] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7232.201587] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7233.349820] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7234.849084] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7237.617944] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7240.080235] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7240.222522] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7241.665310] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7245.166047] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7245.324364] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7247.865121] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7249.449362] LustreError: 328664:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7249.462856] LustreError: 328664:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7251.079211] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7253.840073] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7253.937461] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7256.196945] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7258.030333] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:65) [ 7258.032148] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:33) [ 7258.996716] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0001-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7261.826216] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state (FULL|IDLE) os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid 50 [ 7261.918735] Lustre: DEBUG MARKER: os[cp].lustre-OST0001-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7345.121921] LustreError: lustre-OST0001-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7345.122162] Lustre: lustre-OST0001-osc-MDT0000: Connection to lustre-OST0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7345.124022] LustreError: Skipped 1 previous similar message [ 7345.128241] Lustre: lustre-OST0001: Not available for connect from 0@lo (stopping) [ 7345.131927] Lustre: Skipped 1 previous similar message [ 7349.049050] Lustre: server umount lustre-OST0001 complete [ 7350.244116] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7350.250080] LustreError: 330236:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7351.776925] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7351.780334] Lustre: lustre-OST0000-osc-MDT0001: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7351.784703] Lustre: Skipped 1 previous similar message [ 7351.786461] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7355.361796] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7355.363480] LustreError: 330238:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7355.363748] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7355.364501] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7355.369462] LustreError: 330238:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7356.825757] Lustre: server umount lustre-OST0000 complete [ 7360.481728] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7360.482456] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7360.486079] Lustre: Skipped 1 previous similar message [ 7360.488260] Lustre: Skipped 1 previous similar message [ 7364.318171] Lustre: server umount lustre-MDT0000 complete [ 7365.569285] LustreError: 326967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910656 with bad export cookie 13500960401963529134 [ 7365.572241] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7365.572352] LustreError: 326967:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7365.691372] Lustre: server umount lustre-MDT0001 complete [ 7369.310656] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 7370.365264] Key type lgssc unregistered [ 7370.492482] LNet: 331919:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7370.495169] LNetError: 331919:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7370.501285] LNet: Removed LNI 192.168.201.106@tcp [ 7370.805667] Key type .llcrypt unregistered [ 7370.807209] Key type ._llcrypt unregistered [ 7389.187114] Key type ._llcrypt registered [ 7389.188806] Key type .llcrypt registered [ 7389.227923] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7389.564268] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7389.607552] alg: No test for adler32 (adler32-zlib) [ 7390.468473] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 7390.567391] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 7392.152149] Key type lgssc registered [ 7392.543132] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7395.926153] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7395.939423] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7397.100915] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7398.617677] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7401.625242] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7401.768820] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7403.157224] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7404.172960] Lustre: Modifying parameter general.debug_raw_pointers=Y in log params [ 7406.644681] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7406.766915] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7407.780833] LustreError: 335686:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7407.783550] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:97) [ 7407.787515] LustreError: 335686:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7408.766741] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7411.685442] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7411.766428] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7412.778139] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:65) [ 7413.796646] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7417.977281] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7422.695279] Lustre: Setting parameter general.lod.*.mdt_hash=crush in log params [ 7427.553129] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7427.557186] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7427.563984] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7428.065907] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7428.067213] Lustre: lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7428.073334] Lustre: Skipped 2 previous similar messages [ 7428.326947] Lustre: server umount lustre-MDT0000 complete [ 7429.654324] LustreError: 334179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910720 with bad export cookie 3263508202035295818 [ 7429.656705] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7429.657624] LustreError: 334179:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 1 previous similar message [ 7431.808477] Lustre: server umount lustre-MDT0001 complete [ 7444.074114] Lustre: server umount lustre-OST0000 complete [ 7457.064481] Lustre: server umount lustre-OST0001 complete [ 7471.908420] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7480.007299] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7480.262843] LustreError: 339456:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7480.321052] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7483.595762] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7485.409956] LustreError: 339457:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7489.506318] LustreError: 339456:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7489.568823] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7489.879474] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7492.747477] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7494.590806] Lustre: 340595:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/debug_raw_pointers=Y (mode = 0) failed: rc = -17 [ 7498.878196] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7500.201789] LustreError: 340950:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-OST0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7502.936086] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7505.331940] Lustre: lustre-OST0000: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x280000401:36 to 0x280000401:129) [ 7508.690570] LDISKFS-fs (dm-3): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7508.827089] Lustre: lustre-OST0001: Imperative Recovery not enabled, recovery window 60-180 [ 7508.830891] Lustre: Skipped 1 previous similar message [ 7512.700444] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7514.102193] Lustre: lustre-OST0001: new connection from lustre-MDT0000-mdtlov (cleaning up unused objects from 0x2c0000401:3 to 0x2c0000401:97) [ 7517.217758] Lustre: DEBUG MARKER: Using TIMEOUT=20 [ 7519.495271] Lustre: 342464:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify general/lod.*.mdt_hash=crush (mode = 0) failed: rc = -17 [ 7524.330269] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7524.332104] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7524.338042] Lustre: Skipped 1 previous similar message [ 7524.344289] Lustre: Skipped 3 previous similar messages [ 7527.522590] Lustre: server umount lustre-MDT0000 complete [ 7529.428724] LustreError: 339437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910820 with bad export cookie 3263508202035297470 [ 7529.430498] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7529.438529] LustreError: 339437:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 2 previous similar messages [ 7529.457748] LustreError: 339453:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7529.460080] Lustre: lustre-MDT0001-lwp-OST0001: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7529.474512] LustreError: 339453:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 7529.476671] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7529.476677] Lustre: Skipped 3 previous similar messages [ 7529.513179] Lustre: Skipped 3 previous similar messages [ 7529.638569] Lustre: server umount lustre-MDT0001 complete [ 7537.837791] Lustre: server umount lustre-OST0000 complete [ 7540.060500] Lustre: server umount lustre-OST0001 complete [ 7542.044301] Lustre: DEBUG MARKER: SKIP: conf-sanity test_114 skipping SLOW test 114 [ 7542.892578] Lustre: DEBUG MARKER: SKIP: conf-sanity test_115 skipping excluded test 115 [ 7543.793278] Lustre: DEBUG MARKER: == conf-sanity test 116: big size MDT support ============ 19:40:33 (1788910833) [ 7555.136480] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7559.307822] loop0: detected capacity change from 0 to 1099511627776 [ 7559.615205] SGI XFS with ACLs, security attributes, quota, no debug enabled [ 7559.619881] *** VALIDATE xfs *** [ 7559.625935] XFS (loop0): Mounting V5 Filesystem [ 7559.641823] XFS (loop0): Ending clean mount [ 7560.344812] loop1: detected capacity change from 0 to 18691697672192 [ 7596.597995] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 7602.417089] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7624.482106] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7624.605461] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 7624.622235] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 7624.664517] Lustre: lustre-MDT0000: new disk, initializing [ 7624.704110] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7624.715587] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 7626.893465] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7632.847836] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7632.905757] Lustre: 347593:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7632.949072] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 7632.951887] Lustre: Skipped 1 previous similar message [ 7632.994402] Lustre: lustre-MDT0001: new disk, initializing [ 7633.065623] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 7633.070966] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 7635.418558] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7640.792358] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7643.862493] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7647.623087] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7647.779383] Lustre: lustre-OST0000: new disk, initializing [ 7647.782769] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 7647.789429] Lustre: 349561:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 7649.623273] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 7649.633145] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 7649.689531] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 7651.306915] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7657.030260] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7659.499286] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid 50 [ 7659.598347] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0000.ost_server_uuid in FULL state after 0 sec [ 7662.292675] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing wait_import_state FULL os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid 50 [ 7662.418808] Lustre: DEBUG MARKER: os[cp].lustre-OST0000-osc-MDT0001.ost_server_uuid in FULL state after 0 sec [ 7665.130944] LustreError: lustre-OST0000-osc-MDT0001: operation ost_statfs to node 0@lo failed: rc = -107 [ 7665.134766] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7665.138957] LustreError: Skipped 1 previous similar message [ 7665.140186] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7665.140191] Lustre: Skipped 1 previous similar message [ 7665.146539] Lustre: Skipped 1 previous similar message [ 7669.615251] Lustre: server umount lustre-OST0000 complete [ 7674.336988] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7674.343843] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7674.352542] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 7674.355799] Lustre: Skipped 1 previous similar message [ 7686.112142] Lustre: lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7686.182989] Lustre: server umount lustre-MDT0000 complete [ 7687.604416] LustreError: 348388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788910978 with bad export cookie 3263508202035299024 [ 7687.607613] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7687.609293] LustreError: 348388:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7687.742670] Lustre: server umount lustre-MDT0001 complete [ 7690.859449] XFS (loop0): Unmounting Filesystem [ 7693.090991] Lustre: DEBUG MARKER: == conf-sanity test 117: lctl get_param return errors properly ========================================================== 19:43:02 (1788910982) [ 7696.308156] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7696.557941] LustreError: 352120:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7696.606757] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7696.611667] Lustre: Skipped 2 previous similar messages [ 7698.582171] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7702.508980] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7704.623846] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7707.914402] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7710.251858] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7713.719435] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7717.303922] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7720.871693] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7737.825276] LustreError: lustre-OST0000-osc-MDT0000: operation ost_statfs to node 0@lo failed: rc = -107 [ 7737.827646] Lustre: lustre-OST0000-osc-MDT0000: Connection to lustre-OST0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7737.831444] Lustre: Skipped 1 previous similar message [ 7737.833868] Lustre: lustre-OST0000: Not available for connect from 0@lo (stopping) [ 7737.836068] Lustre: Skipped 6 previous similar messages [ 7748.064104] Lustre: lustre-OST0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 7748.125289] Lustre: server umount lustre-OST0000 complete [ 7754.211031] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 7754.214108] LustreError: Skipped 1 previous similar message [ 7756.000162] Lustre: server umount lustre-MDT0000 complete [ 7757.445246] LustreError: 352099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788911048 with bad export cookie 3263508202035299906 [ 7757.448027] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 7757.449807] LustreError: 352099:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 7757.577240] Lustre: server umount lustre-MDT0001 complete [ 7761.610472] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 7762.781412] Key type lgssc unregistered [ 7762.923508] LNet: 355768:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 7762.926937] LNetError: 355768:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 7762.938426] LNet: Removed LNI 192.168.201.106@tcp [ 7763.308211] Key type .llcrypt unregistered [ 7763.309724] Key type ._llcrypt unregistered [ 7774.846055] Key type ._llcrypt registered [ 7774.847445] Key type .llcrypt registered [ 7774.899550] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 7775.279970] libcfs: HW NUMA nodes: 1, HW CPU cores: 4, npartitions: 1 [ 7775.337622] alg: No test for adler32 (adler32-zlib) [ 7776.211766] Lustre: Lustre: Build Version: 2.17.55_26_gfb964ff [ 7776.318757] LNet: Added LNI 192.168.201.106@tcp [8/256/0/180] [ 7777.904206] Key type lgssc registered [ 7778.325218] Lustre: Echo OBD driver; http://www.lustre.org/ [ 7781.484294] Lustre: lustre-MDT0000: mounting server target with '-t lustre' deprecated, use '-t lustre_tgt' [ 7781.495523] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7782.675423] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 7784.225775] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7787.529845] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7787.687537] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7789.298322] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7792.021418] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 7794.063317] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 7796.640860] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 7796.811208] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 7799.134507] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7801.824262] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 7817.479198] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.6@tcp (stopping) [ 7818.722963] Lustre: lustre-MDT0001-osp-MDT0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 7818.723465] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 7818.723605] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 7818.730290] Lustre: Skipped 1 previous similar message [ 7821.339627] Lustre: server umount lustre-MDT0001 complete [ 7822.591769] LustreError: 357307:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 192.168.201.6@tcp (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7822.598120] LustreError: 357307:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 2 previous similar messages [ 7823.842719] LustreError: 357311:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 7823.849727] LustreError: 357311:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 7825.488706] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 7825.540069] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 7825.542429] Lustre: Found index 1 for lustre-MDT0001, updating log [ 7825.561605] Lustre: 357300:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 7825.705969] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 7827.089872] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 7831.013757] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 7834.597967] LustreError: 361143:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 7834.601184] LustreError: 361143:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 7834.604780] LustreError: 357304:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 7834.624120] LustreError: 361146:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.106@tcp: cfg command failed: rc = -17 [ 7834.627652] Lustre: cmd=cf001 0:lustre-MDT0001-osp-MDT0000 1:osp 2:lustre-MDT0000-mdtlov_UUID [ 7834.627652] [ 8133.091432] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8133.091837] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8133.092183] Lustre: lustre-MDT0001: Not available for connect from 0@lo (stopping) [ 8133.092188] Lustre: Skipped 1 previous similar message [ 8133.095927] Lustre: Skipped 1 previous similar message [ 8136.473200] Lustre: server umount lustre-MDT0001 complete [ 8138.213326] LustreError: 357306:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0001: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8138.218709] LustreError: 357306:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 1 previous similar message [ 8140.305167] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8140.342385] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8140.344571] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8140.359802] Lustre: 357299:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8140.477281] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8141.954332] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 8145.893380] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8146.920304] LustreError: 373464:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8146.922719] LustreError: 373464:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8146.929493] LustreError: 357304:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8146.932281] LustreError: 357304:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8146.951177] LustreError: 373470:0:(obd_config.c:761:class_setup()) osp: device 8 already setup: rc = -17 [ 8146.953740] LustreError: 373470:0:(obd_config.c:866:class_detach()) OBD device 8 still set up [ 8146.955621] LustreError: 373470:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.106@tcp: cfg command failed: rc = -17 [ 8146.958173] Lustre: cmd=cf003 0:lustre-MDT0001-osp-MDT0000 1:lustre-MDT0001_UUID 2:192.168.201.106@tcp [ 8146.958173] [ 8447.237423] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.6@tcp (stopping) [ 8447.238960] Lustre: Skipped 2 previous similar messages [ 8447.969597] LustreError: lustre-MDT0001-osp-MDT0000: operation mds_statfs to node 0@lo failed: rc = -107 [ 8447.971101] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8447.976464] Lustre: Skipped 1 previous similar message [ 8452.350100] Lustre: lustre-MDT0001: Not available for connect from 192.168.201.6@tcp (stopping) [ 8452.352846] Lustre: Skipped 2 previous similar messages [ 8458.208100] Lustre: lustre-MDT0001 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 3. Is it stuck? [ 8458.269613] Lustre: server umount lustre-MDT0001 complete [ 8461.586459] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8461.622962] Lustre: MGS: Regenerating lustre-MDT0001 log by user request: rc = 0 [ 8461.624864] Lustre: Found index 1 for lustre-MDT0001, updating log [ 8461.639389] Lustre: 357302:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8461.754404] Lustre: lustre-MDT0001: Imperative Recovery enabled, recovery window shrunk from 60-180 down to 60-180 [ 8463.019294] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 8466.917697] Lustre: lustre-MDT0001-lwp-OST0000: Connection restored to 0@lo (at 0@lo) [ 8471.526636] LustreError: 386139:0:(obd_mount.c:234:lustre_start_simple()) lustre-MDT0001-lwp-OST0000 attach error -17 [ 8471.529679] LustreError: 386139:0:(tgt_mount.c:675:lustre_lwp_setup()) lustre-MDT0001-lwp-OST0000: setup up failed: rc -17 [ 8471.532777] LustreError: 357304:0:(mgc_request.c:614:do_requeue()) failed processing log: -17 [ 8471.535451] LustreError: 357304:0:(mgc_request.c:614:do_requeue()) Skipped 1 previous similar message [ 8471.552806] LustreError: 386142:0:(lod_lov.c:210:lod_add_device()) lustre-MDT0001-osp-MDT0000: cannot connect to next dev lustre-MDT0001_UUID (-114) [ 8471.557176] LustreError: 386142:0:(obd_config.c:2103:class_config_llog_handler()) MGC192.168.201.106@tcp: cfg command failed: rc = -114 [ 8471.559607] Lustre: cmd=cf014 0:lustre-MDT0000-mdtlov 1:lustre-MDT0001_UUID 2:1 3:1 [ 8471.559607] [ 8779.234472] Lustre: lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8779.234791] LustreError: lustre-MDT0000-osp-MDT0001: operation mds_statfs to node 0@lo failed: rc = -107 [ 8779.237175] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8779.237182] Lustre: Skipped 5 previous similar messages [ 8779.238766] Lustre: Skipped 2 previous similar messages [ 8783.837707] Lustre: server umount lustre-MDT0000 complete [ 8784.354091] LustreError: 357312:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8784.358761] LustreError: 357312:0:(ldlm_lib.c:1192:target_handle_connect()) Skipped 3 previous similar messages [ 8785.094449] LustreError: 357293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788912075 with bad export cookie 11483922027394637314 [ 8785.098688] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8785.099988] LustreError: 357293:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8789.472902] LustreError: 357306:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8789.473215] Lustre: lustre-MDT0001-lwp-OST0000: Connection to lustre-MDT0001 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8790.497047] LustreError: 374824:0:(ldlm_lib.c:1192:target_handle_connect()) lustre-MDT0000: not available for connect from 0@lo (no target). If you are running an HA pair check that the target is mounted on the other server. [ 8791.300049] Lustre: server umount lustre-MDT0001 complete [ 8792.718424] Lustre: server umount lustre-OST0000 complete [ 8795.550271] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 8798.406538] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 8809.191473] Lustre: DEBUG MARKER: == conf-sanity test 120: cross-target rename should not create bad symlinks ========================================================== 20:01:39 (1788912099) [ 8811.303275] LDISKFS-fs (dm-0): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8811.390639] Lustre: Setting parameter lustre-MDT0000.mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity in log lustre-MDT0000 [ 8811.399902] Lustre: ctl-lustre-MDT0000: No data found on store. Initialize space. [ 8811.437585] Lustre: lustre-MDT0000: new disk, initializing [ 8811.456996] Lustre: lustre-MDT0000: Imperative Recovery not enabled, recovery window 60-180 [ 8811.462351] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000200000400-0x0000000240000400]:0:mdt [ 8812.845073] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 8817.126237] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc [ 8817.174107] Lustre: 400759:0:(mgs_llog.c:1450:mgs_modify_param()) MGS: modify lustre-MDT0001/mdt.identity_upcall=/home/green/git/lustre-release/lustre/utils/l_getidentity (mode = 0) failed: rc = -17 [ 8817.199484] Lustre: srv-lustre-MDT0001: No data found on store. Initialize space. [ 8817.201259] Lustre: Skipped 1 previous similar message [ 8817.242272] Lustre: lustre-MDT0001: new disk, initializing [ 8817.281527] Lustre: lustre-MDT0001: Imperative Recovery not enabled, recovery window 60-180 [ 8817.294446] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000240000400-0x0000000280000400]:1:mdt [ 8817.297911] Lustre: cli-ctl-lustre-MDT0001: Allocated super-sequence [0x0000000240000400-0x0000000280000400]:1:mdt] [ 8818.779098] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 8822.653792] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0000-mdc-*.mds_server_uuid 1475 0 [ 8824.613257] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount FULL mdc.lustre-MDT0001-mdc-*.mds_server_uuid 1475 0 [ 8826.978501] LDISKFS-fs (dm-2): mounted filesystem with ordered data mode. Opts: errors=remount-ro,no_mbcache,nodelalloc [ 8827.080763] Lustre: lustre-OST0000: new disk, initializing [ 8827.082423] Lustre: srv-lustre-OST0000: No data found on store. Initialize space. [ 8827.085307] Lustre: 402731:0:(osd_compat.c:1352:osd_object_spec_find()) UNKNOWN COMPAT FID [0x200000001:0x101e:0x0] [ 8827.114719] Lustre: lustre-OST0000: Imperative Recovery not enabled, recovery window 60-180 [ 8829.158474] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_default_debug -1 all [ 8833.160074] Lustre: DEBUG MARKER: oleg106-client.virtnet: executing wait_import_state_mount (FULL|IDLE) osc.lustre-OST0000-osc-[-0-9a-f]*.ost_server_uuid 1475 0 [ 8836.601658] Lustre: ctl-lustre-MDT0000: super-sequence allocation rc = 0 [0x0000000280000400-0x00000002c0000400]:0:ost [ 8836.605047] Lustre: cli-lustre-OST0000-super: Allocated super-sequence [0x0000000280000400-0x00000002c0000400]:0:ost] [ 8836.639113] Lustre: lustre-OST0000-osc-MDT0000: update sequence from 0x100000000 to 0x280000401 [ 8841.699228] Lustre: lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete [ 8841.699973] Lustre: lustre-MDT0000: Not available for connect from 0@lo (stopping) [ 8841.704500] Lustre: Skipped 2 previous similar messages [ 8841.709409] Lustre: Skipped 6 previous similar messages [ 8847.191293] Lustre: server umount lustre-MDT0000 complete [ 8848.382140] LustreError: 400752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) ldlm_cancel from 0@lo arrived at 1788912139 with bad export cookie 11483922027394639253 [ 8848.384571] LustreError: 401568:0:(osp_object.c:618:osp_attr_get()) lustre-MDT0000-osp-MDT0001: osp_attr_get update error [0x200000401:0x2:0x0]: rc = -5 [ 8848.385238] LustreError: MGC192.168.201.106@tcp: Connection to MGS (at 0@lo) was lost; in progress operations using this service will fail [ 8848.386659] LustreError: 400752:0:(ldlm_lockd.c:2564:ldlm_cancel_handler()) Skipped 4 previous similar messages [ 8848.504315] Lustre: server umount lustre-MDT0001 complete [ 8859.990576] Lustre: server umount lustre-OST0000 complete [ 8869.056345] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing set_hostid [ 8872.421636] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing load_modules_local [ 8882.352014] Lustre: DEBUG MARKER: == conf-sanity test complete, duration 8647 sec ========== 20:02:52 (1788912172) [ 8882.851935] Lustre: DEBUG MARKER: === conf-sanity: start cleanup 20:02:52 (1788912172) === [ 8883.903493] Lustre: DEBUG MARKER: === conf-sanity: finish cleanup 20:02:53 (1788912173) === [ 8896.209021] Lustre: DEBUG MARKER: oleg106-server.virtnet: executing unload_modules_local [ 8897.133177] Key type lgssc unregistered [ 8897.253279] LNet: 409083:0:(lib-ptl.c:977:lnet_clear_lazy_portal()) Active lazy portal 0 on exit [ 8897.255302] LNetError: 409083:0:(acceptor.c:252:lnet_acceptor_remove_socket()) Interface ens2 not found [ 8897.263331] LNet: Removed LNI 192.168.201.106@tcp [ 8897.541522] Key type .llcrypt unregistered [ 8897.542732] Key type ._llcrypt unregistered